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Abstract 


Expert  systems  have  great  promise  for  increasing  productivity  and 
effectiveness.  As  budget  cuts  continue  into  the  future,  Air  Force  Civil 
Engineering  will  be  increasingly  concerned  with  its  productivity  and 
effectiveness.  This  thesis  searched  for  Air  Force  Civil  Engineering  expert 
system  applications  using  a  preliminary  selection  criteria  to  discern  the 
knowledge  areas  having  expert  system  potential.  Interviews  were 
conducted  with  experienced  civil  engineers  to  gather  the  ideas. 

The  primary  objective  of  this  thesis  was  to  develop  a  preliminary 
selection  criteria.  Donald  Waterman's  selection  criteria  was  used  as  the 
basis.  The  questions  within  the  selection  criteria  were  reordered  with  the 
most  discriminating  questions  first,  to  eliminate  unfruitful  ideas  quickly. 
Other  discriminating  questions  were  added  to  the  selection  criteria  as 
necessary  for  clarification  and  amplification. 

Eight  experienced  civil  engineers  were  interviewed  during  two 
rounds  of  questioning.  The  first  round  of  interviews  solicited  and  screened 
ideas,  using  the  preliminary  selection  criteria.  The  first  round  generated 
twenty-one  ideas,  which  were  combined  into  fifteen  proposals.  In  the 
second  round,  interviewees  selected  proposals  having  the  greatest  potential 
benefit  to  Air  Force  Civil  Engineering  in  order.  Five  proposals  emerged 
from  the  second  round  of  interviewing:  Job  Order/ Work  Order 
Management,  Design  Schedule  Management,  Beddown  of  New  Aircraft 
Systems,  Facility  Constraints  on  New  Aircraft  Designs,  and  Force 
Development/Force  Structure. 
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HOW  CAN  AIR  FORCE  CIVIL  ENGINEERS  USE  EXPERT  SYSTEMS? 


L  Introduction 


Machines  that  lack  knowledge  seem  doomed  to 
perform  intellectually  trivial  tasks.  Those  that 
embody  knowledge  and  apply  it  skillfully  seem 
capable  of  equaling  or  surpassing  the  best 
performance  of  human  experts. 

IHayes-Roth  and  others,  1983=  31. 


Air  Force  Civil  Engineering  (CE)  is  constantly  faced  with  doing  more 
work  with  fewer  resources.  As  the  national  debt  increases,  and  government 
spending  becomes  tighter,  CE  can  expect  little  or  no  funding  relief  in  the 
near  future.  To  just  stay  "even,"  CE  will  have  to  improve  the  way  they 
manage  and  do  business.  Key  words  for  the  next  decade  will  be 
"effectiveness"  and  "productivity." 

One  avenue  to  productivity  is  through  computers.  CE's  interest  in 
computers  has  been  carried  to  the  point  that  CE  has  designed  its  own 
information  system,  completely  separate  from  Air  Force  data  processing. 

But  to  keep  pace  with  this  trend,  civil  engineers  will  need  to  learn  new 
technologies  and  how  to  implement  them. 
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Artificial  intelligence  offers  one  possible  hope  of  increasing 
productivity  and  improving  effectiveness.  Expert  systems,  a  subset  of 
artificial  intelligence,  are  computer  based  systems  that  capture  knowledge 
and  expertise  within  a  program.  These  programs  differ  from  conventional 
programming  in  that  they  represent  the  knowledge  using  heuristics  versus 
algorithms.  Few  commercial  expert  systems  are  currently  available,  but 
many  are  projected  for  the  near  future.  A  great  variety  of  expert  system 
building  tools  or  shells  are  also  becoming  available.  These  sophisticated 
shells  or  tools  allow  a  comparative  computer  novice  to  develop  a  knowledge 
area  into  an  expert  system  prototype  for  demonstration,  testing,  and 
possible  use. 

One  use  of  expert  systems  would  be  to  "capture"  the  expertise  in 
short  supply  and  distribute  it  to  where  it  is  needed.  Civil  engineering  has 
experts  in  many  different  areas.  Those  experts  know  how  to  approach  the 
problems  and  can  explain  their  solutions.  Experts,  using  expert  systems, 
could  identify  guidelines  for  analyzing  certain  problems  leading  to  their 
solutions. 

This  thesis  examines  the  types  of  CE  applications  that  might  be 
developed  into  expert  systems .  The  thesis  reviews  the  development 
process  of  expert  systems  and  methodologies  for  selecting  suitable 
knowledge  areas  for  development.  It  outlines  a  methodology  for  a 
preliminary  evaluation  of  a  knowledge  area  for  expert  system  development. 

To  gather  information  about  possible  knowledge  areas,  experienced 
civil  engineers  will  be  interviewed  and  solicited  for  suggested  applications. 
The  suggested  applications  will  be  compared  to  a  selection  criteria 
developed  for  expert  systems,  and  a  listing  of  possible  applications  will  be 
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made.  Finally,  the  interviewees  will  be  asked  to  rank  the  listing  of  possible 
applications  in  order  of  need,  recommending  areas  for  expert  system 
application. 

This  thesis  will  be  divided  into  five  chapters.  This  first  chapter 
briefly  acquaints  the  reader  the  thesis  intent.  The  literature  review  will 
cover  the  background  information  leading  up  to  the  statement  of  the 
problem.  The  methodology  will  detail  how  the  research  will  proceed.  The 
results  and  findings  will  present  the  information  gathered  and  list  areas  that 
should  be  considered  for  development.  Lastly,  Chapter  5  will  summarize 
the  research  effort,  discuss  the  conclusions,  and  offer  recommendations  on 
future  research.  Definitions  for  the  terms  used  in  this  thesis  are  available  in 
Appendix  A. 
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II.  Literature  Review 


Overview 

The  purpose  of  this  chapter  is  to  review  the  literature  concerning 
expert  systems,  their  development,  and  how  knowledge  areas  should  be 
selected  for  expert  systems  development.  The  chapter  covers  computer 
development  trends  in  civil  engineering,  a  background  on  expert  systems, 
the  expert  system  development  process,  and  several  problem  selection 
methodologies  for  expert  system  development. 

Trends 

Air  Force  Civil  Engineers  must  find  new  ways  to  increase  their 
productivity  due  to  the  anticipated  yearly  budget  cuts  from  the  Gramm- 
Rudman  deficit  reduction  plan.  Major  General  Clifton  D.  Wright,  former 
Director  of  Engineering  and  Services  HQ  USAF,  identified  increasing  CE 
productivity  as  one  of  the  six  strategic  goals  during  his  tenure  (Astin  and 
Ruff,  1984:  5).  Major  General  George  E.  Ellis  continued  the  emphasis  on 
productivity  by  focusing  on  decentralization  and  greater  computer  usage 
(Sullivan,  1986:  7-8). 

As  the  concern  for  productivity  increased,  so  has  the  interest  in 
computers.  CE  has  used  two  major  computer  systems  in  the  recent  past 
Initially,  CE's  used  the  BEAMS  (Base  Engineer  Automated  Management 
System)  to  provide  database  management.  CE  eventually  outgrew  BEAMS 
and  made  the  transition  to  the  WIMS  (Work  Information  Management 
System). 
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BEAMS.  The  BEAMS  was  originally  implemented  back  in  1968, 
giving  CE  a  management  information  system  (MIS).  The  system  was 
developed  and  run  by  Air  Force  data  processing.  BEAMS  was  a  good  MIS 
for  its  time  but  it  provided  only  standardized  reports,  and  retrieval  of 
specialized  information  was  difficult.  Although  BEAMS  had  some  real  time 
capability,  generally  managers  did  not  have  access  to  it.  Information  was 
batch  loaded  into  the  computer  and  reports  were  generated  overnight  for 
use  by  CE  (Mastrangeli,  1984:  10-11). 

As  BEAMS  matured,  users  became  frustrated  by  the  sy stem's 
inflexibility  and  lack  of  convenience,  and  hence  did  not  make  good  use  of 
the  system.  General  Ellis  summarized  the  situation  best: 


I  (General  Ellis]  didn't  like  (BEAMS]  primarily 
because  it  forced  us  to  manage  in  the  past.  The 
only  thing  BEAMS  let  us  do  was  to  look  back  and 
see  what  it  was  we  hadn’t  done.  I  coined  the 
phrase  "too  late  management."  We  needed  to 
manage  forward.  I  felt  we  could  do  a  much  better 
job  if  we  managed  what  we  had  to  do,  not  what 
hadn't  been  done  (Sullivan,  1983-  5). 


General  Ellis'  response  to  BEAMS  was  to  create  and  support  the  WIMS. 

WIMS.  WIMS  development  started  with  several  'Tiger  Teams," 
groups  of  civil  engineers  working  outside  the  traditional  Air  Force  data 
processing  function  (Holt,  1987).  This  attempt  to  get  into  the  business  the 
data  processing  business  was  met  with  great  resistance  by  Air  Force  data 
processing  initially  and  they  demanded  an  opportunity  to  try  to  meet  CE's 
needs.  When  Air  Force  data  processing  received  the  first  100  typical 
reports  the  Tiger  Teams  wanted,  they  responded  that  it  would  take  2 1 .5 
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man-years  to  accomplish  the  work.  The  Tiger  Teams  continued  with  their 
efforts,  acquired  a  system,  and  put  300  reports  on  their  system  within  60 
days  (Sullivan,  1983:  7). 

The  end  result  of  the  team's  efforts  evolved  into  the  WIMS,  a  user 
developed  decision  support  system.  The  WIMS  system  software  comes 
with  standard  reports,  but  also  has  great  flexibility  to  generate  new  reports. 
It  interfaces  with  BEAMS  files,  so  previously  recorded  data  is  available. 
WIMS  also  provides  a  real  time  capability  with  terminal  access  for  CE 
(Mastrangeli,  1984:  12-14). 

Rivard  and  Huff  have  noted  that  more  and  more  user's  are 
developing  their  own  applications: 


To  users,  most  of  the  advantages  of  [user 
developed  applications!  are  related  to  the  ultimate 
involvement  of  the  user  in  the  development 
process.  Since  users  do  not  have  to  translate  and 
communicate  their  information  needs  to  outsiders, 
the  problems  inherent  in  determining  information 
requirements  are  reduced  or  eliminated  [Rivard 
and  Huff,  1984: 401. 


User  involvement  was  the  key  part  in  the  development  of  WIMS.  The  users 
understood  their  requirements  best  and  were  able  to  search  and  find  their 
own  solutions  (Holt,  1987). 

The  trend  toward  user  developed  systems  also  seems  to  be  evolving 
in  the  realm  of  expert  systems. 

It  is  of  the  utmost  importance  for  any  civil 
engineer  who  wishes  to  build  an  expert  system  to 
realize  that  one  can  learn  to  be  knowledge 
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engineer  rather  rapidly,  in  fact  many  civil 
engineers  are  already  knowledge  engineers 
without  even  knowing  it.  [Ludvigsen  and  others, 
1986:  281. 


One  of  the  contributing  factors  toward  users  developing  expert 
systems  seems  to  be  the  ease  of  using  expert  system  languages. 


Experienced  users  of  microcomputer  spreadsheets 
and  databases  have  been  able  to  make  the 
transition  to  these  expert  system  languages  easily. 
The  author  has  taught  two  day  short  courses  in 
which  persons  with  almost  no  prior  computer 
background  have  been  able  to  build  simple 
prototype  expert  systems  using  [an  expert  system 
tool)  with  only  limited  assistance  .  .  .  [Levitt, 
1986:  63] 


Expert  Systems 

Defined.  Donald  Waterman  in  his  book,  A  Guide  to  Expert  Systems 
describes  an  expert  system  as  "a  computer  program  using  expert  knowledge 
to  attain  high  levels  of  performance  in  a  narrow  problem  area"  (Waterman, 
1986: 11).  Expert  systems  vary  from  conventional  programs  in  a  number  of 
ways  (see  Table  1 ).  Conventional  programs  generally  solve  problems  using 
algorithmic  techniques  to  manipulate  data.  Expert  systems  differ  by 
manipulating  a  knowledge  base  using  heuristic  methods  dealing  with 
ambiguous  and  incomplete  information.  Expert  systems  typically  solve 
problems  in  the  following  categories:  interpretation,  prediction,  diagnosis, 
debugging,  design,  planning,  monitoring,  repair,  instruction,  and  control 
(Hayes-Roth,  1983: 5). 
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Table  1 

Conventional  Programs  and  Expert  Systems  Characteristics 


Conventional  Programs _ 

Representation  and  use  of  data 
Knowledge  and  control  integrated 
Algorithmic  (repetitive)  process 
Effective  manipulation  of  large  data  bases 

Programmer  must  ensure  uniqueness 
completeness 

Midrun  explanation  impossible 


Expert  Systems _ 

Representation  and  use  of  knowledge 

Knowledge  and  control  separated 

Heuristic  (inferential)  process 

Effective  manipulation  of  large 
knowledge  bases 

Knowledge  engineer  inevitably  and 
relaxes  uniqueness  and  completeness 
constraint 

Midrun  explanation  desirable  and 
achievable 


Oriented  toward  numerical  processing _ Oriented  toward  symbolic  processing 

(Maher,  1987:4) 


Nature  of  Expertise.  To  better  grasp  expert  systems,  the  expertise 
underlying  the  system  needs  to  be  understood  more  fully.  Paul  E.  Johnson 
defines  an  expert 


An  expert  is  a  person  who  because  of  training  and 
experience  is  able  to  do  things  the  rest  of  us 
cannot;  experts  are  not  only  proficient  but  also 
smooth  and  efficient  in  the  actions  they  take. 
Experts  know  a  great  many  things  and  have  tricks 
and  caveats  for  applying  what  they  know  to 
problems  and  tasks;  they  are  also  good  at  plowing 
through  irrelevant  information  in  order  to  get  to 
the  basic  issues,  and  they  are  good  at  recognizing 
problems  they  face  as  instances  of  types  with 
which  they  are  familiar.  Underlying  the  behavior 
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of  experts  is  the  body  of  operative  knowledge  we 
have  termed  expertise  (Johnson,  1983;  781. 


Experts  are  also  said  to  have  the  following  attributes: 

1.  Eiperts  have  the  ability  to  solve  problems. 

2.  Experts  can  explain  their  solutions. 

3.  Experts  learn  by  experience. 

4.  Experts  can  restructure  their  knowledge 

5.  Experts  know  when  to  break  the  rules. 

6.  Experts  can  determine  the  relevance  of  information. 

7.  Experts  exhibit  graceful  degradation  of  performance. 

Of  the  seven  attributes  experts  demonstrate,  expert  systems  can  only 
partially  demonstrate  the  first  three  (Shurkin,  1983:  75). 

Roles.  Expert  systems  can  function  in  four  roles  in  an  organization,  a 

consultancy  role,"  a  "checklist  role,"  a  "refining  expertise  role,"  and  a 
"training  role  "  (Basden,  1984:  63-64). 

Consultant.  Using  an  expert  system  as  a  consultant,  the  non¬ 
specialist  can  obtain  counsel,  guidance,  or  information  from  expert  systems 
similar  to  the  specialist.  Such  a  system  not  only  frees  the  specialist,  but 
increases  the  non -specialist  s  access  to  the  needed  expertise  (Basden,  1984: 
63-64,  Allen,  1986:9). 

Checklist.  In  the  checklist  role,  the  eipert  system  would 
question  the  user  about  a  subject  and  lead  the  user  to  the  same  conclusion 
as  the  expert  would  derive.  Experts  systems  can  intelligently  order  the 
questions,  avoid  irrelevant  questions,  and  rapidly  arrive  at  the  solution.  The 
system  might  also  provide  documentation  of  the  consultation  for  future 
reference  (Basden,  1984: 66). 

Refining  Expertise.  Most  experts  will  admit  to  gaps  in  certain 
parts  their  knowledge.  In  creating  an  expert  system,  the  knowledge  must 
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be  defined  and  mapped,  which  in  turn  identifies  the  gaps  in  knowledge. 
Experts  systems  can  “be  of  significant  benefit  as  a  guide  to  research,  by 
highlighting  the  weaknesses  in  current  understanding"  (Basden,  1984:  67). 

Training.  The  training  rote  is  often  seen  as  an  area  of  great 

potential.  Key  personnel  using  an  expert  system  can  learn  processes  and 
decisions  which  represent  expert  knowledge. 

As  a  training  device  the  expert  system  provides 
new  staff  members  with  a  vast  reservoir  of 
experiences  and  strategies  from  which  to  learn 
about  recommended  policies  and  methods.  The 
system  can  also  be  adapted  to  train  novices  in 
specific  tasks,  such  as  claims  adjusting  or  financial 
planning  [Waterman,  1986:  7-81. 

Components.  Expert  systems  are  generally  composed  of  four 
components:  the  knowledge  base,  the  inference  engine,  the  user  interface, 
and  the  development  engine.  Figure  1  shows  the  interrelation  between  the 
components.  The  knowledge  base  is  a  database  of  static  data  plus  relational 
information.  The  inference  engine  actually  manipulates  the  knowledge 
base  using  analyses,  forming  hypotheses,  and  auditing  the  processes  per 
some  strategy.  The  user  interface  refers  to  the  terminal  connecting  the 
user  to  the  system.  The  development  engine  allows  the  knowledge 
engineer  to  create,  modify,  add,  or  delete  information  from  the  knowledge 
base  (Wolfgram  and  others,  1987: 13-15). 

In  addition  to  these  components,  expert  systems  may  also  have  an 
explanation  facility.  This  facility  can  be  as  simple  as  tracing  the  path  of 
execution  through  the  knowledge  base  or  as  complex  as  explaining 
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Basic  Components  of  an  Expert  System 
(Wolfgram  and  others,  1987: 13) 


the  reasoning  and  justification  behind  each  step  of  the  process.  An 
explanation  facility  allows  the  user  to  derive  deeper  understanding  into  the 
subject  in  question  as  well  as  gain  confidence  in  the  system  (Maher,  1987: 
7, 24). 

Development  of  expert  systems  is  possible  in  many  different 
programming  environments.  The  expert  system  can  be  directly  coded  into 
structured  computer  languages  such  as  C,  FORTRAN,  or  Pascal  or  into 
languages  specifically  developed  for  artificial  intelligence  such  as  LISP  or 
PROLOG.  Expert  system  development  tools  are  also  available  to  assist  the 
knowledge  engineer  in  development  of  the  expert  system.  These  tools 
speed  the  development  by  removing  the  drudgery  and  debugging 
associated  with  programming  the  development  and  inference  engines 
(Maher,  1987:  15-18). 

Benefits.  Expert  systems  have  a  number  of  benefits.  First,  as  the 
expert  system  is  developed,  the  underlying  expert  s  knowledge  is  made 
apparent.  “Hence  a  written  record  of  the  knowledge  of  a  domain  is 
frequently  made  available  for  the  first  time"  (Allen,  1986:  22). 

Second,  expert  systems  do  not  forget  relevant  factors  under  stress  or 
in  a  time-critical  situation,  thus  yielding  greater  consistency  over  human 
decision  makers.  Expert  systems  are  impartial,  giving  "the  same  answer  to 
beggars  and  kings"  (Sell,  1985: 15).  Weak  decisions  made  on  the  basis  of 
politics  or  partiality  are  avoided.  Expert  systems  are  also  unaffected  by  the 
time  of  day,  and  do  not  "suffer  from  Monday  morning  blues  or  Friday 
afternoon  impatience"  (Sell,  1985:  15,  Allen,  1986: 22). 

Large  quantities  of  details  and  tedious  tasks  are  difficult  even  for  the 
most  experienced  expert  To  help  an  expert,  an  expert  system  "can 
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assimilate  huge  amounts  of  data  and  examine  a  large  number  of  of  related 
factors  simultaneously"  (McCain,  1987:  9).  Also,  as  the  methodologies  for 
solving  the  problems  change,  an  expert  system  can  "quickly  assimilate  new 
information  and  apply  the  information  to  aid  management  in  the  decision¬ 
making  process"  (McCain,  1987:  9). 

Expert  systems  will  also  help  middle  managers  to  perform  their  work. 
"During  the  last  few  years,  training  developers  have  learned  to  make  a 
sharp  distinction  between  knowledge  a  performer  needs  to  memorize  and 
knowledge  a  person  can  access  by  means  of  a  job  or  decision  aid"  (Harmon 
and  King,  1985: 219-220).  Many  decisions  learned  and  made  by  middle 
managers  do  not  require  memorization  of  the  expertise,  only  familiarity 
with  the  concepts  and  the  system.  Expert  systems  can  relieve  the  middle 
managers  from  the  tedium  of  knowing  vast  amounts  of  details  required  for 
their  jobs  and  free  them  for  more  productive  tasks  (Harmon  and  King,  1985: 
219-220). 

Drawbacks.  Expert  systems  do  have  certain  drawbacks  which  limit 
their  applicability.  Once  designed,  an  expert  system  is  not  as  creative  as  a 
human  would  be.  "Human  experts  handle  unexpected  events  by  using 
imaginative  and  novel  approaches  to  problem  solving,  including  drawing 
analogies  to  situations  in  completely  different  domains.  Programs  have  had 
little  success  doing  this"  (Waterman,  1986:  14). 

Human  experts  gain  much  of  their  information  directly  through  the 
use  of  their  physical  senses.  For  computers  to  "view"  the  data,  the 
information  must  be  translated  into  symbols  understood  by  the  computer. 
Information  is  sometimes  lost  in  translation,  possibly  constraining  the 
computer's  range  of  solutions.  Because  of  computers  limited  sensory 
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abilities,  experts  systems  are  generally  limited  to  problem  areas  dealing 
with  cognitive  or  reasoning  skills  only  (Waterman,  1986:  14). 

Humans  generally  have  what  is  termed  "commonsense." 
Commonsense  is  a  broad  area  of  knowledge  about  the  world  and  how  it 
works.  Commonsense  knowledge  encompasses  a  great  deal  of  information 
which  would  be  very  difficult  to  include  in  an  expert  system.  It  also  tells  us 
what  we  do  not  know.  An  expert  system  might  try  in  a  futile  effort  to 
answer  a  question  there  is  no  solution  for,  or  worse  yet,  pose  an  impossible 
solution  to  an  unsuspecting  user.  An  expert  system  s  advice  and  counsel 
should  not  stand  alone,  but  be  tempered  with  human  judgement 
(Waterman,  1986:  15). 

Examples.  Many  different  types  of  systems  are  being  prototyped  and 
tested.  Included  in  this  section  are  several  examples  of  systems  being 
anticipated  currently. 

DSCAS.  The  Differing  Site  Condition  Analysis  System  models  a 
lawyer  s  decision  process  in  analyzing  a  contractor  s  claim  of  a  differing  site 
condition.  A  differing  site  condition  generally  refers  to  a  situation  costing 
the  contractor  additional  time  and  effort  that  he  could  not  have  foreseen. 
Approximately  20%  of  the  contract  modifications  issued  by  the  Air  Force  are 
attributed  to  differing  site  conditions  (Osgood,  1988).  Often,  claims  are 
settled  long  after  the  completion  of  the  work,  because  of  the  intense  and 
complex  coordination  required.  The  program  is  intended  to  provide 
contract  administrators  job  site  access  to  the  legal  expertise  needed  in 
clarifying  the  claims.  The  user  steps  through  a  series  of  22  modules 
considering  such  elements  as  the  claim's  timeliness,  evaluating  express- 
implied  contract  conditions,  and  determining  superior  knowledge.  By 
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providing  the  legal  expertise  on  site,  the  contract  administrators  can  decide 
whether  the  claim  has  merit  and  whether  or  not  to  negotiate 
(Kruppenbacher,  1984:2-3,  151). 

Inactive  Hazardous  Waste  Site  Characterization.  This  system 

uses  the  U.S.  Environmental  Protection  Agency’s  Hazard  Ranking  System 
(HRS)  to  rank  hazardous  waste  sites.  The  HRS  ranks  the  potential  hazard 
of  the  sites  in  three  categories;  (1)  migration  of  pollutants  through 
groundwater,  surface  water,  and  air,  (2)  explosion  and  fire  potential,  and  (3) 
direct  contact  with  hazardous  pollutants.  Data  inputs  include  such  items  as 
the  soil  permeability,  soil  stratification,  groundwater  flow,  and  gradient. 

The  system  produces  a  HRS  score  for  site  permeability  and  groundwater 
flow  (Law,  1986:  159-166). 

Construction  Schedule  Analysis.  This  system  provides  an 
analysis  and  evaluation  of  a  construction  schedule  from  the  facility's  owner 
perspective.  Data  used  for  the  analysis  is  gathered  from  the  project 
management  system  (PRIMAVERA™)  and  the  database  management  system 
(dBaselll™)  using  an  expert  system  shell.  As  construction  progresses  and 
the  schedule  is  revised,  the  system  evaluates  the  schedule  in  a  number  of 
areas.  For  example,  built-up  roofing  should  not  be  scheduled  during  winter 
months  because  the  outside  air  temperatures  are  expected  below  the 
minimum  required.  The  system  also  notes  higher/iower  production  rates 
than  anticipated  and  identifies  other  activities  effected  (O’Connor,  1986: 67- 
71). 
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Expert  System  Development 


Expert  system  builders  don't  have  a  series  of  well 
defined  steps  that  they  follow  when  constructing  a 
system.  The  inherent  complexity  of  the  system 
building  process  precludes  laying  out  all  the  steps 
in  advance.  As  a  result,  system  builders  have 
found  that  an  evolutionary  development  style  is 
the  most  effective  way  to  proceed  [Waterman, 
1986:  1351. 


Three  development  processes  are  described  in  this  section.  The 
Hayes-Roth  scheme  shows  a  classical  development  of  an  expert  system. 
Harmon  and  King  provide  the  element  of  scale  in  their  two  expert  system 
developments,  detailing  both  a  large  scale  and  small  scale  development 
scheme. 

Hayes-Roth.  The  evolution  of  an  eipert  system  involves  several 
stages  including  identification,  conceptualization,  formalization, 
implementation,  testing,  and  finally  the  prototype  revision.  Figure  2  shows 
the  progression  of  the  stages. 

Identification  Stage.  During  this  stage,  the  problem  is  explored 
from  several  different  aspects.  The  participants  (experts  and  knowledge 
engineers)  are  selected  and  their  roles  in  the  development  effort  are 
defined.  The  domain  expert  and  the  knowledge  engineer  attempt  to 
characterize  the  problem.  The  terms  for  the  project  are  clarified  and  key 
concepts  delineated  through  repeated  interaction  between  the  knowledge 
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Stages  of  Knowledge  Acquisition 
(Hayes-Roth  and  others,  1983: 139) 


engineer  and  the  domain  expert.  Resources  such  as  computers,  funding, 
and  time  of  the  domain  expert  and  knowledge  engineer,  are  considered  and 
allocated.  The  system  goals  are  set  forth  (Hayes-Roth  and  others,  1983; 
104-143). 

Conceptualization  Stage,  key  concepts  and  relations  are  made 
explicit  under  this  stage  by  continued  interaction  between  the  domain 
expert  and  the  knowledge  engineer.  The  knowledge  engineer  may  start 
building  a  prototype  to  test  the  concepts  delineated  by  the  domain  expert. 
Specific  examples  are  used  to  challenge  the  system.  The  knowledge 
engineer  may  also  elect  not  to  show  the  system  to  the  domain  expert  for 
fear  of  interfering  with  the  next  stage,  formalization  (Hayes-Roth  and 
others,  1983:  143-144). 

Formalization  Stage.  The  formalization  stage  converts  the  key 
concepts,  subproblems,  and  information  flow  characteristics  into  formal 
representations.  Formalizing  the  concepts  determines  how  the  expert 
system  will  generate  hypotheses .  A  knowledge-engineering  tool  or  frame 
work  is  selected  for  the  prototype  (Hayes-Roth  and  others,  1983;  144-146). 

Implementation  Stage.  The  formalized  knowledge  is  mapped 

into  the  representational  framework.  Inconsistencies  in  the  system  are 
worked  out.  The  end  product  is  an  expert  system  prototype  ready  for 
testing  (Hayes-Roth  and  others,  1983:  144-147). 

Testing  Stage.  The  testing  stage  evaluates  the  system.  Two  or 

three  typical  cases  are  run  through  the  system  to  assure  its  performance. 
Then  atypical  cases  are  used  to  challenge  the  system  to  make  the  strengths 
and  weaknesses  of  the  program  become  more  apparent.  Ultimately,  the 


program  is  assessed  in  light  of  the  original  goals  and  judged  to  be  either 
adequate  or  inadequate  (Hayes-Roth  and  others,  1983:  147-148). 

Prototype  Revision.  The  expert  system  is  constantly  revised 

and  altered  during  the  building  process.  This  revision  process  does  not  end 
with  an  adequate  prototype  but  continues  with  refinements,  redesigns,  and 
reformulations  of  the  implemented  system.  "The  result  of  revision  should 
be  a  convergence  of  performance,  once  the  expert  system  s  scope  of 
reasoning  has  been  stabilized"  (Hayes-Roth  and  others,  1983: 149).  If  the 
performance  does  not  converge,  then  more  drastic  revisions  may  be 
necessary  (Hayes-Roth  and  others,  1983: 148-149). 

Harmon  and  King.  Harmon  and  King  add  the  dimension  of  size  to 

their  view  of  the  development  process.  At  one  end  of  the  spectrum  are 
large-scale  systems  encompassing  2000  or  more  rules.  At  the  other  end  of 
the  spectrum  are  small-scale  systems  with  100  to  500  rules.  Harmon  and 
King  advocate  different  development  schemes  for  each  end  of  the  spectrum 
(Harmon  and  King,  1985, 228). 

Large-Scale  Knowledge  System  Development  Harmon  and 

King  see  large-scale  knowledge  system  development  as  a  team  effort 
incorporating  a  knowledge  engineer  and  the  domain  expert  similar  to 
Hayes-Roth  scheme.  They  suggest  a  slightly  different  development  path, 
presented  in  phases: 

Phase  I.  Selection  of  an  Appropriate  Problem. 

Phase  II.  Development  of  a  Prototype  System. 

Phase  III.  Development  of  a  Complete  Expert  System. 

Phase  IV.  Evaluation  of  the  System. 

Phase  V.  Integration  of  the  System. 

Phase  VI.  Maintenance  of  the  System. 
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Phase  I  -  Selection  of  an  Appropriate  Problem.  In  this 


phase,  the  problem  domain  and  task  are  clearly  identified.  "Large-scale 
systems,  because  of  the  very  large  initial  development  costs,  must 
necessarily  focus  on  problems  that  are  carefully  selected  to  assure  a  large 
and  rapid  payback  for  their  developers"  (Harmon  and  King,  1985: 197). 

The  expert  must  be  identified  and  witling  to  provide  the  expertise.  A 
tentative  approach  to  the  problem  is  formulated,  along  with  a  cost  and 
benefits  analysis.  Before  moving  on  to  the  next  phase,  a  specific  plan  is 
proposed  to  guide  the  development  of  the  system  (Harmon  and  King,  1985: 
197-201). 

Phase  II  -  Development  of  a  Prototype  System.  The 
knowledge  engineer  accomplishes  a  series  of  tasks  to  develop  the  prototype. 
Initially,  the  knowledge  engineer  gathers  information  about  the  domain  and 
the  task.  The  expert  provides  four  or  five  typical  cases  for  the  system  to 
solve.  The  knowledge  engineer  works  through  the  cases  with  the  expert, 
learning  the  problem  solving  strategies  and  heuristics  related.  From  this 
information,  the  knowledge  engineer  creates  a  prototype.  Performance 
criteria  for  evaluating  the  prototype  is  established.  The  knowledge 
engineer  and  the  expert  jointly  test  the  prototype  using  a  variety  of  cases. 
The  knowledge  engineer  and  the  expert  revise  and  modify  the  system  until 
it  is  functioning  satisfactorily.  The  knowledge  engineer  then  develops  a 
detailed  design  for  the  complete  system  (Harmon  and  King,  1985: 201-203). 

Phase  HI  -  Development  of  a  Complete  Expert  System. 

At  this  point,  the  knowledge  engineer  may  consider  discarding  the 
prototype  in  favor  of  rethinking  the  design.  By  rethinking  the  design, 
economies  of  effort  can  be  realized  for  both  the  user  of  the  system  and  the 
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system  itself.  Also,  additional  heuristics,  developed  as  rules,  are  added  to 
deal  with  the  subtler  aspects  of  certain  problems.  The  interface  is  tailored 
to  be  easy  and  natural  for  the  user  by  providing  explanations  or  defining  the 
terms  of  the  system  (Harmon  and  King,  1985:  203-205). 

Phase  IV  -  Evaluation  of  the  System.  The  expert  and 

knowledge  engineer  field  test  the  completed  expert  system  against  the 
performance  criteria  established  in  Phase  II.  Other  experts  view  the  system 
and  challenge  it  with  examples  of  their  own  (Harmon  and  King,  1985:  205). 

Phase  V  ^  Integration  of  of  the  System.  During  this 

phase,  the  system  is  integrated  into  the  work  environment.  Users  are 
trained  on  how  to  use  the  system  and  with  develop  confidence  in  the 
system.  The  knowledge  engineer  fades  from  the  picture  during  this  phase 
(Harmon  and  King,  1985:  205-206). 

Phase  VI  -  Maintenance  of  the  System.  Asthe 
knowledge  and  expertise  for  accomplishing  the  job  of  the  expert  system 
change,  the  system  must  be  updated  and  kept  current.  If  the  modifications 
are  simple  enough  and  the  system  is  friendly  enough,  the  expert  or  manager 
in  charge  of  the  system  can  make  the  changes  (Harmon  and  King,  1985: 
206-207). 

Small-Scale  Knowledge  System  Development  The  small-scale 
knowledge  system  as  envisioned  by  Harmon  and  King  is  different  from  the 
concepts  presented  so  far. 

Many  American  companies  are  attempting  to 
reduce  their  layers  of  middle  managers.  .  .  . 

Expert  systems  will  probably  be  developed  to 
assist  middle  managers  who  are  being  asked  to 
monitor  a  large  and  increasing  volume  of 
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information.  The  systems  will  help  gather  and 
rearrange  the  information.  Moreover,  they  will 
provide  managers  with  tools  that  will  help  them 
explore  the  implications  of  fast-breaking 
developments.  Such  managerial  systems,  probably 
packaged  into  managerial  workstations,  may  begin 
to  appear  in  large  corporations  within  the  next 
five  years  (Harmon  and  King,  1985: 216). 


Not  only  will  the  systems  be  used  by  middle  management,  in  many 
cases,  the  knowledge  systems  will  be  developed  by  middle  management. 


Our  own  view  is  that  small  knowledge  system 
building  tools  can  and  will  be  used  by  middle 
managers  and  training  developers  to  solve  a  vast 
array  of  small  irksome  problems.  The  individuals 
using  these  tools  will  not  be  “knowledge 
engineers"  but  will,  instead,  be  people  who  are 
close  to  the  problems.  Senior  application 
examiners  will  develop  small  knowledge  systems 
that  will  provide  assistance  to  clerical  personnel 
(Harmon  and  King,  1985: 1781. 


Harmon  and  King  describe  a  modified  development  path  for  the 
smaller  system  in  steps  (shown  in  Table  2).  While  the  steps  are  similar  to 
the  two  previous  schemes  presented,  the  knowledge  engineer  s  job  has 
essentially  been  replaced  by  the  expert  and  the  tool  or  shell  used  to  derive 
the  expert  system.  Knowledge  systems  will  be  created  by  individuals  that 
are  actually  using  them,  with  a  minimum  of  training.  These  knowledge 
systems  will  move  some  simpler  decisions  to  lower  levels,  giving  middle 
managers  more  time  for  complex  and  crucial  decisions  (Harmon  and  King, 
1985: 194). 
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Table  2 


DEVELOPING  A  SMALL  KNOWLEDGE  SYSTEM 

Step  1 

Select  a  tool  and  implicitly  commit  yourself  to  a 
particular  consultation  paradigm. 

Step  2 

Identify  a  problem  and  then  analyze  the  knowledge  to  be 
included  in  the  system 

Step  3 

Design  the  system.  Initially  this  involves  describing  the 
system  on  paper.  It  typically  involves  making  flow 
diagrams  and  matrices  and  drafting  a  few  rules. 

Step  4 

Develop  a  prototype  of  the  system  using  the  tool.  This 
involves  actually  creating  the  knowledge  base  and  testing 
it  by  running  a  number  of  consultations. 

Step  5 

Expand,  test,  and  revise  the  system  until  it  does  what  you 
want  it  to  do. 

Step  6 

Maintain  and  update  the  systems  needed. 

(Harmon  &  King,  1985:  178) 


Moreover,  we  think  their  appearance  will  be 
welcomed  in  the  same  way  that  managers 
welcomed  electronic  spreadsheet  programs. 
Individuals  throughout  large  organizations  will 
begin  to  document  the  knowledge  that  is  actually 
used  to  get  the  job  done  [Harmon  and  King,  1985: 
1941 


Ultimately,  Harmon  and  King  see  the  small  systems  as  a  way  for 
organizations  to  build  confidence  in  constructing  and  using  expert  systems. 
Developing  small  systems  initially  avoids  the  very  large  development  costs, 
but  verifies  the  concept  of  an  expert  system  to  the  organization  (Harmon 
and  King,  1985=  197). 
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Problem  Selection 


All  of  the  different  development  schemes  involve  considerable  time 
and  effort,  which  point  to  the  importance  of  proper  problem  selection.  "The 
choosing  of  the  domain  is  a  critical  task  in  the  development  of  an  expert 
system”  (Prerau,  1985: 26). 

The  selection  criteria  of  the  problem  area  for  expert  system 
development  gives  tremendous  insight  into  expert  systems.  The  capabilities 
and  limitations  of  systems  can  be  seen  clearly.  Novices  can  team  from 
different  the  selection  criteria  to  develop  proper  expectations  for  using 
expert  systems. 

Waterman.  Waterman  suggests  considering  three  main  areas  before 
selecting  a  problem  to  be  developed  into  an  expert  system.  Consider  expert 
systems  only  if  expert  system  development  is  possible,  justified,  and 
appropriate"  (Waterman,  1986:  127). 

Possible.  In  considering  a  problem  area,  Waterman  lists  eight 
different  requirements  that  must  all  be  met  to  make  expert  system 
development  possible,  shown  in  Figure  3.  "One  of  the  most  important 
requirements  is  that  genuine  experts  exist”  (Waterman,  1986:  128).  If 
genuine  experts  do  not  exist,  the  system  will  be  difficult,  at  best,  to  develop. 
Beyond  having  genuine  experts,  the  experts  must  agree  on  the  solutions  and 
be  able  to  articulate  their  methods  for  solving  the  problems.  If  the  expert 
cannot  sufficiently  describe  his  technique,  the  knowledge  will  be  difficult  to 
extract  and  develop.  Disagreement  between  the  experts  on  major  tenets  of 
the  problem  and  solutions  will  also  impede  system  development  (Waterman, 
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Waterman  also  considers  the  requirements  for  solving  the  task.  The 
task  must  use  reasoning,  or  cognitive  skills  only.  Tasks  requiring 
specialized  sensory  abilities,  such  as  a  wine  connoisseur  s  refined  ability  to 
smell,  would  be  difficult  to  develop  into  an  expert  system.  Also,  tasks 
requiring  common  sense  beyond  the  user's  ability  present  an  impediment  to 
development  (Waterman,  1986:  128). 

In  general,  the  task  must  not  be  to  difficult.  If  an  expert  cannot  teach 
a  novice  the  skill  required  for  the  task,  it  is  unlikely  that  the  expert  could 
teach  a  computer.  “Or  if  any  expert  takes  days  and  weeks  rather  than  hours 
to  solve  the  problem,  there's  a  good  chance  that  it's  too  difficult  or  too 
complex  for  a  knowledge  engineering  approach"  (Waterman,  1986: 128). 
Complementing  the  expert  s  ability  to  articulate  his  methods,  the  task  must 
be  well  understood.  “If  the  task  is  so  new  or  poorly  understood  that  it 
requires  basic  research  to  find  solutions,  knowledge  engineering  will  not 
work"  (Waterman,  1986:  128-129). 

Justified.  Any  one  of  several  situations  can  justify  the  effort 

and  eipense  of  developing  an  expert  system,  as  shown  in  Figure  4.  If  a  task 
has  a  very  high  payoff,  an  organization  can  justify  an  expert  system 
development.  The  payoff  can  be  in  either  time  or  money.  Another  situation 
is  where  human  expertise  is  being  lost  to  retirement  or  personnel  changes. 
An  expert  system  could  be  used  to  capture  the  expertise  and  retain  for 
future  use  as  well  as  distributing  the  knowledge  to  where  it  is  needed 
(Waterman,  1986.  130-131). 
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Figure  4 

Justification  for  Expert  System  Development 
(Waterman,  1986: 130) 
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An  expert  system  development  is  justified  when  the  expertise  is 
scarce  or  needed  at  many  different  locations  at  the  same  time. 


The  problem  is  compounded  when  the  company 
needs  similar  expertise  at  many  different  locations, 
such  as  process  control  expertise  for  each 
distillation  column  owned  by  a  petrochemical 
plant  This  generates  a  need  for  multiple  versions 
of  the  expert,  something  that  can  be  done  easily 
and  at  virtually  no  cost  when  the  expert  is  a 
computer  program  [Waterman,  1986:  131]. 


Lastly,  an  expert  system  development  is  justified  when  the  expert  is 
needed  in  hostile  or  unfriendly  environment  Using  an  expert  system  in  a 
hostile  situation  risks  the  system  and  not  the  expert  (Waterman,  1986:  131 ). 

Appropriate.  Three  factors  must  be  met  in  deciding  whether  a 

problem  area  is  appropriate:  nature  of  the  area,  the  complexity  of  the  area, 
and  the  scope  of  the  problem  to  addressed  by  the  expert  system.  Figure  5 
shows  these  characteristics. 

Nature.  Problems  most  appropriate  for  expert  system 
development  are  heuristic  in  nature.  Heuristic  problems  use  strategies  and 
rules  of  thumb  to  achieve  acceptable  solutions.  Mathematically  intense 
algorithms,  yielding  the  optimum  solutions,  would  not  be  good  candidates 
for  expert  systems. 


5 


* 


In  some  sense,  the  expert  systems  approach  is  the 
last  resort.  If  the  problem  can  be  solved 
mathematically  or  with  clever  algorithms,  then 
those  methods  should  be  used.  If  it's  to  difficult 
for  these  conventional  techniques,  expert  systems 
may  be  appropriate  (Waterman,  1986: 132).  fc 
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Figures 


Characteristics  Appropriate  to  Expert  System  Development 

(Waterman,  1986: 132) 


Compleiity.  The  problem  addressed  by  the  expert 
system  should  not  be  too  easy.  Much  effort  and  time  are  required  to 
develop  an  expert  system.  If  an  easy  problem  is  selected,  then  the  users 
may  quickly  learn  the  heuristics  and  disregard  the  system.  Unless  the 
intent  of  the  building  the  system  is  for  training,  a  more  complex  problem 
should  normally  be  considered  (Waterman,  1986:  132). 

Scope.  The  scope  of  the  problem  needs  to  be  defined  in 
terms  of  practical  value  and  manageable  size. 

It  should  be  sufficiently  narrow  to  make  the 
problem  manageable  and  sufficiently  broad  to 
ensure  that  the  problem  has  some  practical 
interest.  Unfortunately,  the  definitions  of 
manageable  and  practical  depend  and  the 
particular  problem  domain.  And  to  make  matters 
worse,  choosing  the  proper  scope  is  crucial  to  the 
success  of  the  expert  system  endeavor  {Waterman, 

1986:  1331. 

Typically,  the  knowledge  engineer  divides  a  large  problem  down  into  parts 
until  the  area  to  be  developed  is  a  manageable  size,  but  still  has  a  practical 
value  (Waterman,  1986=  133-134). 

Cross.  Artificial  intelligence  is  viewed  by  many  as  being  some  sort  of 
science  fiction  or  managerial  panacea.  A  myriad  of  expectations  develop 
when  artificial  intelligence  is  discussed.  User's  and  expert's  expectations  of 
expert  systems  are  often  unrealistic  (Cross,  1988). 

Solutions.  Managers  and  experts  frequently  expect  expert 

systems  to  solve  problems  that  cannot  be  solved  now.  Unfortunately,  this  is 
not  the  case.  Unless  an  expert  can  solve  the  problem,  an  expert  system  will 
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not  be  of  assistance.  The  current  state  of  expert  system  development  is 
dependent  on  reviewing  cases  of  situations  that  have  been  solved  to  develop 
the  system.  If  no  satisfactory  solutions  have  been  obtained  up  to  this  point 
in  time,  the  most  an  expert  system  development  might  contribute  is  a 
clarification  of  the  problem  solving  technique  or  methodology  (Cross,  1988). 

Volatility.  Because  of  the  time  and  effort  needed  to  modify  an 

expert  system,  the  knowledge  area  being  developed  needs  to  be  fairly 
mature  and  closer  to  static  than  dynamic.  If  major  changes  are  being  made 
in  the  techniques  for  solving  problems,  an  expert  system  would  be  difficult 
to  develop.  One  way  to  determine  the  volatility  of  the  knowledge  area  is  to 
look  back  three  to  five  years  and  compare  the  technology  for  solving  the 
problems  at  that  time  with  those  of  the  present.  If  major  changes  have 
been  made  in  the  recent  past,  changes  may  also  be  projected  for  the  near 
future  (Cross,  1988). 

Prerau.  David  Prerau  presents  a  very  good  methodology  in  his  article 
"Selection  of  an  Appropriate  Domain  of  an  Expert  System."  Many  of 
Waterman's  salient  points  are  covered,  but  organized  in  a  different  fashion: 

Basic  Requirements 

Type  of  Problem 

The  Expert 

Problem  Bounds 

Domain  Area  Personnel  -  Users 

Other  Desirable  Features 

The  article  includes  other  crucial  points  that  must  be  considered  before 
embarking  on  an  expert  system  development.  Prerau's  selection  criteria  is 
in  Appendix  A  (Prerau,  1985:  28). 
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Commitment.  Before  starting  on  the  development  of  an  expert 
system,  Prerau  suggests  soliciting  and  establishing  support  for  the  effort 
Foremost,  senior  management  must  commit  resources  and  time  to  see  the 
project  through  development.  Also,  the  expert  must  be  committed  to  the 
development  of  the  project.  If  any  of  the  participants  waver,  the 
development  effort  may  flounder  and  the  expert  system  may  be  weak  and 
nearly  useless  (Prerau,  1985:  28-29). 

User  support  must  also  be  solicited  and  nurtured.  An  expert  system 
has  little  benefit  unless  used.  Realistic  user  expectations  of  the  systems 
utility  and  capability  need  to  be  cultivated.  The  users  need  to  understand 
that  “even  a  successful  system  will  likely  be  limited  in  scooe,  and  like  a 
human  expert,  may  not  produce  optimal  of  correct  results  100*  of  the  time" 
(Prerau,  1985:  28).  During  the  development  of  the  expert  system,  the  users 
need  to  be  consulted  to  ensure  that  the  domain  and  the  scope  of  the  system 
have  utility  for  them  (Prerau,  1985:  28-29). 

Political  Sensitivity.  Prerau  also  suggests  considering  the 

politically  sensitivity  toward  the  development  of  an  expert  system  within  an 
organization.  Tor  example,  there  may  be  certain  practice,  embodied  in 
heuristics,  which  may  prove  embarrassing  if  written  down,  such  as  how 
certain  customers  are  treated  relative  to  other  customers"  (Prerau,  1985: 

29).  Factions  within  the  organization  may  also  challenge  the  system  if  it 
does  not  produce  results  that  favor  them  politically.  If  a  expert  system's 
intent  is  to  allocate  funds  to  acquire  resources,  the  validity  of  the  system 
may  constantly  be  challenged  (Prerau,  1985:  29). 
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Statement  of  the  Problem 

The  problem  is,  simply  stated,  "How  can  Air  Force  Civil  Engineers 
use  expert  systems?"  To  answer  the  question,  ideas  must  be  gathered  and 
compared  with  a  criteria  to  select  the  viable  options.  The  ideas  wilt  be 
solicited  from  experienced  civil  engineers  and  evaluated  with  their 
assistance.  Those  providing  the  ideas,  though  they  might  have  a  tacit 
interest  in  the  subject,  will  most  likely  be  under  the  crush  of  other 
impending  work.  Hence,  the  criteria  used  to  niter  the  ideas  must  not  only 
be  complete,  but  also  succinct.  This  thesis  creates  a  methodology  for 
screening  potential  ideas,  and  selecting  justified,  appropriate,  and  possible 
ideas. 

Prerau  s  selection  criteria,  though  very  complete,  is  also  very  long 
containing  53  questions.  Many  of  the  questions  concern  soliciting  or 
confirming  commitment  by  all  parties  involved,  prior  to  the  final 
commitment  of  resources.  While  Prerau's  methodology  is  appropriate 
before  making  the  last  step,  it  would  be  difficult  to  employ  in  preliminary 
survey  searching  for  ideas  and  proposals. 

The  methodology  this  thesis  developed  is  a  modification  of 
Waterman's  selection  criteria.  Waterman  s  criteria  will  be  used  in  total,  but 
reordered  with  the  most  discriminating  questions  first,  to  eliminate  unviable 
ideas  quickly.  Also,  additional  discriminating  questions  from  other  selection 
criteria  will  be  added  when  appropriate. 
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III.  Methodology 


Overview 

This  chapter  discusses  the  methodology  underlying  this  thesis. 
Included  in  this  chapter  are  sections  covering  why  interviewing  was  chosen 
versus  written  survey  to  identify  ideas,  how  the  structured  questionnaire 
was  derived,  how  the  interviews  were  conducted,  and  how  the  analysis  and 
conclusions  will  be  presented. 

Why  Interviewing? 

The  researcher  interviewed  experienced  civil  engineers  to  gather 
ideas  for  possible  development  into  expert  systems.  Interviewing  provided 
a  means  of  interacting  with  the  expert,  to  clarify  and  further  develop 
information  supplied  by  the  interviewee.  Because  of  the  relative  newness 
of  the  field,  those  being  interviewed  were  unfamiliar  with  expert  systems 
and  their  capabilities.  A  personal  interview  created  the  opportunity  to 
respond  to  the  interviewee  s  questions  firsthand. 

Interviewing  does  have  several  drawbacks.  Because  of  the  time  it 
takes  to  accomplish  an  interview,  a  fewer  number  of  interviews  can  be 
accomplished  in  the  same  time  that  it  would  take  to  complete  written 
surveys.  Also,  the  interview  is  generally  limited  to  a  certain  block  of  time; 
whereas,  a  survey  could  be  read,  thought  about,  and  then  accomplished  at  a 
later  time. 

The  benefits  of  interviewing  outweigh  its  drawbacks.  Interviewing 
commits  both  the  interviewee  and  the  researcher  to  spending  adequate  time 
on  the  subject  and  providing  a  better  quality  of  information  than  a  written 
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survey.  Furthermore,  interviewing  offers  an  opportunity  to  approach  the 
problem  from  the  interviewee's  point  of  view  and  is  not  limited  to  the 
researcher  perspective.  Ultimately,  the  goal  of  this  thesis  is  to  gather  new 
ideas,  and  interviewing  offers  the  best  means  of  accomplishing  that  goat 
(Davis,  1987). 

Research  Methodology 

The  research  methodology  has  been  divided  into  seven  steps:  ( 1 ) 
familiarization  with  expert  systems,  (2)  development  of  the  selection 
criteria,  (3)  the  first  round  of  interviewing,  (4)  compilation  the  ideas,  (5)  the 
second  round  of  interviewing,  (6)  results  and  findings,  and  (7)  recom¬ 
mendations  on  the  selection  criteria. 

Step  1_  -  Familiarization  with  Expert  Systems.  The  researcher 

became  familiar  with  expert  systems,  their  development,  and  several 
problem  selection  criteria.  The  familiarization  was  accomplished  by  a 
literature  review  and  discussion  with  experts  in  the  field  of  expert  systems. 

Step  2  ^  Development  Selection  Criteria.  When  interviewing  top- 

level  managers,  time  is  at  a  premium.  The  objective  was  to  minimize  the 
time  taken  to  accomplish  the  survey,  while  still  attempting  to  obtain  as 
many  ideas  as  possible.  To  meet  this  objective,  the  questions  were  ordered 
with  the  most  discriminating  questions  first.  By  ordering  the  interview  this 
way,  ideas  with  low  potential  for  expert  system  development  were  dropped 
as  soon  as  possible. 

The  underlying  basis  for  this  research's  selection  criteria  was 
Waterman's  methodology.  Waterman  s  methodology  was  supplemented  by 
inputs  from  Cross  and  Prerau.  Figure  6  graphically  shows  the  selection 


Figure  6 

Preliminary  Selection  Criteria 


1 1 .  Does  the  task  have 
a  practical  value? 


Figure  6  -  continued 
Preliminary  Selection  Criteria 
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criteria,  which  is  also  described  in  Appendix  C.  All  the  questions  in  the 
selection  criteria  were  conceived  to  be  answered  either  yes'  or  'no'. 

Waterman.  Waterman  s  selection  criteria  was  used  in  total. 

The  appropriate  and  possible  sections  were  distributed  across  the  expertise, 
task,  and  other  consideration  sections  of  the  selection  criteria.  The  justified 
section  was  left  in  tact,  except  that  the  concerns  about  expertise  being  lost 
and  scarce  will  be  combined  into  Question  4  (Waterman,  1986:  127-129). 

Cross.  The  first  two  questions  in  the  selection  criteria  were 

derived  from  Cross'  considerations  of  a  problem  having  a  solution  and 
concerning  the  volatility  of  the  knowledge  area  being  explored.  These  were 
considered  the  most  discriminating  questions.  Cross  also  contributed 
Question  14,  "Do  the  problems  encountered  share  some  of  the  same  common 
characteristics?"  (Cross,  1988). 

Prerau.  A  number  of  questions  were  drawn  from  Prerau's 

selection  criteria  that  either  amplified  certain  points  or  filled  in  possible 
gaps.  For  example,  Question  9  "Are  experts  better  than  novices  at 
performing  the  task?"  amplifies  Question  7  "Do  genuine  expert  exist?”  If 
experts  are  no  better  than  novices,  it  is  difficult  to  declare  that  genuine 
experts  exist  (Prerau,  1985.  26-30). 

The  question  order  in  the  selection  criteria  was  developed  from 
interviews  with  Major  Stephen  Cross  and  Major  James  Holt.  The  first 
section,  thought  to  be  the  most  discriminating,  was  whether  the  ideas 
proposed  were  realistic  or  not.  The  next  most  discriminating  section  was 
the  justification  of  the  idea,  with  the  experts,  task  and  other  considerations 
sections  following.  Selection  criteria  was  tested  against  a  surrogate 
interviewee  as  to  insure  consistency  (Cross,  1988,  Holt,  1988). 
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Step  3  ^  First  Round  of  Interviews.  Eight  experienced  military  civil 
engineers  were  selected  to  be  interviewed.  This  group  was  chosen  on  the 
basis  of  their  experience  in  civil  engineering.  The  number  of  individuals 
was  limited  by  the  time  available  to  the  researcher.  The  individuals  chosen 
are  listed  in  Appendix  B. 

Before  starting  the  questioning,  the  interviewees  were  shown  a 
simple  expert  system  called  the  "Wine  Advisor"  (Giarratano,  1987).  The 
interviewees  were  requested  to  think  of  a  meal  they  would  have  wine  with, 
and  the  expert  system  assisted  them  in  selecting  a  wine.  The  system 
demonstrated  to  the  interviewees  the  capabilities  of  an  expert  system  first¬ 
hand.  A  brief  background  of  expert  systems  was  discussed  and  several 
examples  were  presented. 

Initially,  background  information  was  exchanged  to  acquaint  both 
the  researcher  and  interviewee  with  each  other.  The  interviewee  then 
selected  a  problem  area  within  civil  engineering  for  expert  system 
application.  The  questioning  and  discussion  continued  through  the  first 
idea,  regardless  of  its  potential.  By  reviewing  all  the  questions  with  the 
first  idea,  the  interviewee  became  more  familiar  with  the  types  of  questions 
and  would  be  more  discriminating  of  the  ideas  he  suggested  after  the  first, 
idea  Starting  with  the  second  idea  proposed,  ideas  were  eliminated  by 
more  than  one  unacceptable  response.  It  was  hoped  that  each  interviewee 
could  supply  five  ideas  to  be  run  through  the  problem  selection  process, 
though  the  limits  of  time  were  understood. 

Step  4  -  Compiling  the  Ideas.  After  the  interviews  were 
completed,  the  ideas  were  compiled  into  a  master  listing  of  proposals. 
Similar  ideas,  if  any,  were  grouped  together  into  single  proposals.  The 
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master  listing  will  be  reviewed  by  the  interviewees  for  their  inputs  in 
Step  5.  None  of  the  proposals  were  attributed  to  their  contributing 
interviewee,  so  other  interviewees  selection  of  a  proposal  was  based  purely 
on  the  proposal's  merit.  Also,  interviewees  were  not  allowed  to  select  their 
own  proposals,  unless  their  idea  was  shared  with  another  interviewee.  The 
intent  of  this  limitation  was  to  eliminate  each  of  the  interviewees  selecting 
only  their  proposals. 

Step  5  -  Second  Round  of  Interviews.  The  interviewees  were 
requested  to  select  five  proposals.  Their  selections  were  based  on  their 
experience  and  in  their  view,  which  proposals  would  benefit  Air  Force  Civil 
Engineering  the  most.  Those  selections  were  then  rank  ordered. 

Step  6  -  Results  and  Findings.  The  selections  of  interviewee  were 

weighted  as  follows: 

Priority  Weight 

1  5  points 

2  4  points 

3  3  points 

4  2  points 

5  I  points 

The  five  selections  with  the  heaviest  weightings  are  described  more  fully  in 
Chapter  4  and  recommended  for  consideration  of  development  into  expert 
systems.  The  descriptions  have  been  expanded  from  the  information 
gathered  during  the  interviews. 

Step  7  ^  Recommendations  on  Methodology.  Recommendations  for 
changes  and  improvement  in  the  methodology  are  suggested  in  Chapter  5  of 
this  thesis. 
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IV.  Results  and  Findings 


Overview 

This  chapter  presents  the  ideas  collected  through  the  first  round  of 
interviews  using  the  selection  criteria  and  the  findings  of  the  second  round 
selections. 

First  Round  Ideas 

Interviews.  Eight  experienced  managers  in  Air  Force  Civil 
Engineering  were  interviewed  in  accordance  with  Step  3  in  the  research 
methodology.  The  list  of  interviewees  is  in  Appendix  D.  Twenty-one  ideas 
were  collected  from  the  interviewees.  A  summary  of  the  ideas  collected  is 
presented  in  Table  3.  The  raw  data  collected,  ’yes’/no/maybe'  answers  to 
each  of  the  questions  in  the  selection  methodology,  is  listed  in  Appendix  E. 
Uncertainty.  The  original  intent  of  the  questions  in  the  methodology 

was  to  elicit  'yes'/'no'  responses  from  the  interviewees.  Some  uncertainty 
was  expected,  but  not  quite  on  the  scale  experienced.  The  uncertainty  was 
incorporated  by  allowing  'yesVnoV'maybe'  responses.  The  uncertainty  in 
the  responses  reflects  of  the  initial  nature  of  the  preliminary  screening 
process.  As  topics  are  selected  and  screened  further  using  Preraus 
selection  criteria,  involving  experts  and  users  alike,  the  uncertainty  should 
be  reduced  (Prerau,  1985:  26-30). 

Selection  Criteria  Applied.  Ideas  from  the  interviewees  were  only 

eliminated  during  the  interviews  if  the  underlying  Waterman  selection 
criteria  or  Cross'  questions  were  violated.  Because  of  the  uncertainty 
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present,  a  broad  interpretation  was  taken  of  the  ideas  collected  in  an  effort 
to  include  all  possibilities.  Ideas  offered  by  the  interviewees  were  made 
into  “proposals"  for  explanation  in  this  chapter  and  use  during  the  second 
round  of  interviews.  Similar  ideas  were  combined  for  presentation  into 
single  proposal. 

Proposals 

From  the  twenty-one  ideas  collected  in  the  interviews,  fifteen 
different  proposals  were  derived. 

JL  Classification  of  Job  Orders/Work  Orders 

Description.  As  conceived,  this  expert  system  would  assist  in 
classifying  job  orders  and  work  orders  per  guidance  provided  by  regulation 
and  base  policy.  The  system  would  be  used  by  new  clerks  learning  the 
process.  Clerks  typically  make  the  initial  classification  of  incoming  job 
orders  and  work  orders,  subject  to  the  approval  of  the  section  chief.  More 
accurate  initial  classifications  would  ease  the  work  load  on  the  section  chief. 
The  expert  system,  by  the  training  the  clerk,  would  relieve  the  section  chief 
further.  The  expert  system  could  possibly  be  integrated  into  the  existing 
civil  engineering  computer,  WIMS  (Work  Information  Management  System) 
or  the  next  generation  of  civil  engineering  computer  systems . 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  ail  the  selection  criteria. 

2.  Force  Beddown  at  a  Bare  Base. 

Description.  This  expert  system  would  assist  the  civil 
engineers  in  bedding  down  aircraft  and  personnel  at  a  bare  base.  The 
system  would  provide  a  flexible  checklist  that  could  be  reordered  to  reflect 
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the  priorities  of  the  situation.  The  system  might  also  include  elements  such 
as  project  scheduling,  inventory  control,  and  suggested  layouts  of  base 
facilities  to  minimize  damage  from  attacks. 

Proposed  by.  Three  interviewees. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 

Realistic:  The  second  interviewee  thought  the  system 
was  in  a  volatile  knowledge  area  because  of  the  changing  requirements  for 
Prime  BEEF  (Base  Engineering  Emergency  Force). 

Expertise:  The  second  interviewee  felt  that  experts  may 

not  come  to  agreement  on  all  solutions. 

Task:  The  first  interviewee  was  unsure  that  the  task 

could  be  manipulated  symbolically.  He  felt  there  may  be  a  lack  of  relevant 
test  cases.  The  second  interviewee  saw  each  beddown  operation  as  a 
separate  situation,  not  sharing  many  characteristics.  He  also  felt  that  the 
task  is  not  well  understood  and  may  not  be  of  a  manageable  size  for  an 
expert  system. 

Other:  The  second  interviewee  felt  that  the  task  may 

require  skills  other  than  cognitive. 

3.  Selection  of  a  Minimum  Operating  Strip  (MOS). 

Description.  This  expert  system  would  help  civil  engineers 
select  the  minimum  operating  strip  (MOS)  to  launch  and  recover  aircraft 
after  a  bomb  attack.  Data  inputs  to  the  system  would  include  such  items  as 
locations  of  bomb  damage  and  live  ordnance.  The  system  could  be  used 
during  peacetime  exercises  to  increase  the  proficiency  of  civil  engineers  as 
well  as  during  wartime. 
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Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exception: 

Task:  Interviewee  thought  a  conventional  programming 
technique  or  computer  aided  design  (CAD)  might  be  easier  to  employ  than 
an  expert  system. 

4.  Force  Development/Force  Structure. 

Description.  Under  this  premise,  an  expert  system  would  be 
devised  to  structure  civil  engineering  forces  going  to  war.  Such  items  as 
training  levels  and  experience  of  units  may  be  included  in  such  a  system. 
Once  developed,  the  system  would  be  used  by  headquarters  to  plan 
operations  and  exercises. 

Proposed  by.  Two  interviewees. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 

Realistic:  The  first  interviewee  felt  that  the  system  may 
not  be  realistic  because  of  recent  changes  in  policy  concerning  the 
composition  of  Prime  BEEF  teams. 

Expertise:  The  first  interviewee  felt  no  genuine  experts 

may  exist,  and  that  if  experts  existed,  they  may  not  agree  on  the  solutions. 

Task-.  The  first  interviewee  was  also  concerned  the  task 

may  not  be  understood  sufficiently  to  be  developed  into  an  expert  system 
and  may  be  beyond  a  manageable  size.  The  second  interviewee  saw  the 
process  as  more  algorithmic  than  heuristic. 
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5.  Facility  Constraints  on  New  Aircraft  Designs. 

Description.  This  expert  system  would  delineate  existing 
facility  constraints  to  design  managers  of  new  aircraft  systems.  Constraints 
would  include  such  items  as  facility  dimensions,  safe  distance  for  munitions, 
and  perhaps  even  long-range  environmental  effects.  Financial  impacts 
could  be  reflected  in  rough  cost  estimates  to  provide  facilities  for  system 
that  exceed  the  current  facilities  capacity. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 

Expertise:  The  interviewee  was  unsure  all  the  experts 

would  agree  on  the  solutions  presented  by  the  system. 

Task:  The  interviewee  felt  the  system  might  actually  be 

a  combination  of  heuristics  and  algorithms.  Also,  the  interviewee  was 
unsure  whether  the  task  could  be  taught  to  novices,  since  many  of  the 
facility  limitations  are  learned  by  experience. 

6.  Corrosion  Control. 

Description.  A  corrosion  control  expert  system  would  advise 
users  on  inspection  and  maintenance  of  corrosive  surfaces.  Surface 
preparations  and  paint  types  would  be  identified  depending  on  the  material 
and  use.  The  system  could  be  updated  periodically  to  reflect  technological 
improvements  available  though  different  materials. 

Proposed  by.  Two  interviewees. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 
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Realistic:  The  second  interviewee  thought  of 
technological  improvements  volatile  enough  to  overcome  developing  an 
expert  system. 

Expertise:  The  second  interviewee  was  uncertain 

whether  all  the  experts  would  agree  on  the  solutions  to  all  problems. 

Task:  The  second  interviewee  was  unsure  whether  the 

task  could  be  taught  to  novices  or  if  it  was  of  a  manageable  size. 

Other:  The  first  interviewee  was  uncertain  that 

inspection  could  be  totally  classified  as  a  cognitive  skill. 

7.  Design  Schedule  Management. 

Description.  This  expert  system  would  assist  in  design 
schedule  management.  The  system  would  be  able  to  assess  all  the  impacts 
of  changes  in  schedules,  priorities,  and  projects.  The  system  could  also 
include  such  checklist  functions  as  assuring  asbestos  removal  are 
considered  during  the  design  phase  on  existing  facilities.  The  system  could 
possibly  encompass  the  programming  process  as  well. 

Proposed  by.  Two  interviewees. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 

Expertise:  Both  interviewees  were  unsure  that  the 

experts  would  all  agree  on  the  solutions  presented  by  such  a  system. 

Task:  The  second  interviewee  was  uncertain  the  task  of 

managing  design  was  well  understood. 

8.  Vehicle  Allocation. 

Description.  A  vehicle  allocation  expert  system  would  be  used 
by  a  vehicle  officer  within  a  civil  engineering  squadron  to  allocate  vehicles. 
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Several  different  strategies  might  be  available  on  such  a  system  depending 
on  location,  mission,  and  vehicle  availability. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  ail  selection  criteria  with  the  following 

exception: 

Expertise:  The  interviewee  was  unsure  whether 

genuine  experts  existed. 

9.  Energy  Management. 

Description.  The  expert  system  in  this  example  would  help  the 
base  energy  management  czar.  The  system  would  identify  various 
strategies  to  employ  conservation  methods.  The  system  could  be  updated 
periodically  to  include  technological  improvements. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  ail  selection  criteria  with  the  following 

exception: 

Task:  The  interviewee  was  concerned  the  process  might 

be  more  algorithmic  than  heuristic. 

10.  Job  Order/Work  Order  Management. 

Description.  This  expert  system  would  interface  with  the 
WIMS  database  to  improve  the  efficiency  of  scheduling  and  material 
ordering.  Information  from  recent  work  could  be  compiled  to  provide 
production  estimates  and  lead  times  on  materials.  The  system  could  also 
suggest  alternatives  when  problems  are  encountered  in  accomplishing 
scheduled  work,  such  as  listing  another  job  in  close  proximity  or  scheduled 
maintenance. 

Proposed  by.  Two  interviewees. 
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Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions 

Task:  The  first  interviewee  thought  there  might  be 
some  conventional  system  that  would  HU  the  need  as  well  as  an  expert 
system. 

Other:  The  first  interviewee  thought  that  such  an  expert 

system  might  require  skills  beyond  cognitive  skills. 

11.  Well-Rounded  People  Advisor. 

Description.  This  expert  system  would  advise  senior 
leadership  on  jobs  and  training  for  junior  and  mid- level  managers.  The 
system  would  gather  data  pertaining  to  the  individual  s  background, 
education,  and  experience.  The  system  would  suggest  a  particular  job  and 
training  to  insure  an  individual  was  well-rounded.  The  system  might  be 
structured  to  handle  people  individually,  or  make  recommendations  for  a 
group. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  ail  selection  criteria  with  the  following 

exceptions: 

Expertise:  The  interviewee  was  unsure  as  to  whether 

the  experts  would  agree  on  how  to  make  a  well-rounded  officer. 

Task:  The  interviewee  did  not  think  the  task  was  well 

understood  nor  could  it  be  taught  to  novices. 

12.  Liquid  Fuel  System  Maintenance. 

Description.  In  this  case,  the  expert  system  would  train  airman 
on  the  maintenance  of  liquid  fuel  systems.  The  system  would  also  include 
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safety  advisories  and  could  possibly  serve  as  a  checklist  prior  to  the  start  of 
maintenance. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exception: 

Task:  The  interviewee  thought  the  training  process 

might  be  more  algorithmic  than  heuristic. 

13.  Beddown  of  New  Aircraft  Systems. 

Description.  This  expert  system  would  be  compiled  from 
previous  lessons-learned  during  the  beddown  of  new  aircraft  systems.  In 
the  future,  as  new  systems  are  bedded  down,  the  system  could  be  updated 
and  revised  to  reflect  the  new  lessons-learned. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 

Expertise:  The  interviewee  was  uncertain  genuine 

experts  exist  and  whether  the  experts  could  articulate  their  methods. 

Task:  The  interviewee  was  also  unsure  whether  the 

task  was  of  a  manageable  size. 

14.  Groundwater  Decontamination. 

Description.  This  expert  system  would  advise  an  engineer  on 

the  most  effective  pumping  scheme  for  treating  a  contaminated  aquifer. 
Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exceptions: 


50 


Realistic:  The  interviewee  felt  that  the  knowledge  area 


is  in  a  state  of  flux. 

Expertise:  The  interviewee  was  uncertain  all  of  the 

experts  would  agree  on  common  solutions. 

Task:  The  interviewee  was  unsure  about  whether  the 

task  was  well  understood  or  if  the  skill  could  be  taught  to  novices. 

15.  Scheduling  and  Assignment  of  Engineers. 

Description.  This  expert  system  would  advise  managers  on 
assigning  engineers  to  design  projects  based  on  their  current  schedule, 
experience,  and  training.  The  system  might  also  be  expanded  to  other  areas 
such  as  programming  and  construction  management. 

Proposed  by.  One  interviewee. 

Selection  Criteria.  Met  all  selection  criteria  with  the  following 

exception: 

Other:  The  interviewee  was  uncertain  whether  the  task 
is  difficult  enough  to  require  an  expert. 

Second  Round  Selections 

The  eight  interviewees  were  asked  to  review  the  fifteen  proposed 
expert  systems.  Each  of  the  interviewees  were  asked  to  select  five 
proposals  based  on  their  experience  in  civil  engineering.  After  making  their 
selections,  they  were  requested  to  prioritize  the  selections  from  one  to  five, 
with  one  being  their  top  selection. 

Points  were  assigned  to  each  of  the  selections,  per  Step  6  in  the 
research  methodology.  The  results  of  the  survey  are  compiled  in  Figure  7. 
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Weighted  Selection  Results 


The  top  selections  from  the  survey  were: 

1.  Proposal  No.  10  -  Job  Order/ Work  Order  Management. 

2.  Proposal  No.  7  -  Design  Schedule  Management. 

3.  Proposal  No.  14  -  Beddown  of  New  Aircraft  Systems. 

4.  Proposal  No.  5  -  Facility  Constraints  on  New  Aircraft  Designs. 

5.  Proposal  No.  4  -  Force  Development/Force  Structure. 

These  five  proposals  do  not  exclude  the  other  proposals  as  potential  expert 
systems  or  from  eventual  development  They  do  reflect  the  expert  systems 
that  would  possibly  be  highest  in  demand.  Also,  further  definition  of  the 
expert  systems  with  the  experts  and  the  users  needs  to  be  done  prior  to 
development 

Selected  Proposals 

Each  of  the  selected  proposals  is  further  defined  below. 

Job  Order/Work  Order  Management.  An  expert  system  that  handles 

job  order  and  work  order  management  would  be  most  effective  if 
developed  for  the  WIMS  or  the  next  generation  of  civil  engineering 
computer  systems  and  be  used  Air  Force  wide.  The  scope  of  the  system,  as 
presented,  is  quite  broad  and  might  make  development  difficult  The 
system  will  need  to  be  constructed  in  modules,  to  make  each  module 
sufficiently  narrow  enough  to  develop  (Zody,  1988,  Wise,  1988). 

The  expert  system  would  gather  information  about  production  rates, 
material  availabilities,  and  the  job  orders  and  work  orders  to  be 
accomplished.  The  system  would  also  schedule  the  work  per  the  work 
classification,  status  of  materials,  manpower  availability,  and  user 
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discretion.  The  expert  system  would  be  generic,  but  elements  of  the 
program  could  be  customized  for  individuals  bases.  Program  elements 
such  as  the  priorities  of  different  classifications,  the  percentage  of 
scheduled  versus  unscheduled  work,  and  preventive  maintenance  coutd  be 
changed  at  the  base  level  (Zody,  1988,  Wise,  1988). 

Beyond  scheduling,  the  system  might  flag  bottlenecks  in  the  flow  of 
work  and  materials  so  managers  could  work  to  relieve  the  problems.  As 
part  of  the  scheduling  process,  the  expert  system  coutd  automatically 
develop  alternate  schedules  based  on  possible  different  contingencies,  for 
example,  weather  or  exercises  (Zody,  1988,  Wise,  1988). 

Design  Schedule  Management.  This  expert  system  is  similar  to  the 

job  order /work  order  expert  system,  because  it  would  probably  be 
beneficial  on  an  Air  Force  wide  level.  This  expert  system  probably  best 
implemented  on  WIMS  or  the  next  generation  of  civil  engineering  computer 
systems  so  the  expert  system  could  access  data  directly  (Lollis,  1988,  Wise, 
1988). 

Ideally,  the  system  would  provide  assistance  with  managing  projects 
starting  in  programming  and  continuing  until  construction  is  complete.  The 
system  could  be  modularized  to  handle  projects  in  the  different  stages. 
Impacts  of  not  accomplishing  projects  could  be  listed  with  the  projects.  As 
the  schedules  change,  the  impacts  could  be  compared  so  that  managers  are 
able  to  make  the  most  prudent  solutions  (Lollis,  1988,  Wise,  1988). 

Expert's  strategies  for  managing  project  funding  and  design 
schedules  could  be  elicited  and  captured  within  the  expert  system. 

Differing  strategies  depending  on  political  and  funding  environments  could 
be  available  for  the  user  on  the  system.  Elements  of  the  system  might  be 
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changed  so  that  it  could  be  customized  to  suit  the  user  s  preference  (Lollis, 
1988,  Wise,  1988). 

Beddown  of  New  Aircraft  Systems.  This  proposal  differs  from  the 
first  two  in  that  it  would  only  be  useful  at  the  headquarters  and  the  bases 
responsible  for  a  new  system  beddown.  The  knowledge  base  would 
provide  nearly  all  the  information  and  inferences  needed  to  run  the  eipert 
system;  hence,  the  system  could  be  developed  on  a  personal  computer, 
making  it  easily  transportable  (Zody,  1988). 

The  knowledge  base  would  be  captured  from  individuals  that  have 
managed  a  beddown  of  a  new  system  and  documentation  from  previous 
beddowns.  The  knowledge  could  be  divided  in  various  sections  such  as 
dimensional  interfaces,  mechanical  and  electrical  interfaces,  munitions, 
fuels,  environmental  concerns,  etc..  As  new  and  differing  problems  come 
up,  they  could  be  entered  directly  into  the  knowledge  base  to  update  it 
(Zody,  1988). 

As  facility  designers  develop  plans  for  the  beddown  of  the  new 
aircraft  system,  the  expert  system  could  highlight  previous  problems  and 
solutions.  One  of  the  interviewees  provided  the  following  example.  When 
the  F-16s  replaced  the  F-4s  in  Europe,  there  was  a  difference  in  the  angle 
of  the  jet  exhaust,  which  was  not  planned  for  by  the  facility  designers. 
USAFE  requires  that  the  aircraft  engines  be  run  up  in  the  shelter. 
Ultimately,  a  new  design  and  change  order  had  to  be  issued  to  modify  the 
shelters  for  the  aircraft.  An  expert  system  with  this  knowledge  may  help 
prevent  design  problems  in  the  future  (Zody,  1988). 

Facility  Constraints  on  New  Aircraft  Designs.  This  proposal  is  very 

similar  to  the  last  with  the  exception  that  it  is  intended  for  the  design 
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managers  of  new  aircraft  systems.  The  knowledge  base  would  be  captured 
from  individuals  that  have  managed  a  beddown  of  a  new  system  and 
documentation  from  previous  beddowns.  It  differs  from  the  previous 
system  by  providing  information  intended  to  assist  the  aircraft  designer, 
and  not  the  facility  designer.  Similar  to  the  previous  system,  the  different 
interfaces  between  the  aircraft  and  the  facilities  would  be  detailed  (Lollis, 
1988). 

As  the  design  manager  uses  the  system,  the  impact  of  exceeding  the 
current  facilities'  constraints  could  be  estimated  in  dollars.  For  example,  if  a 
new  Tighter  exceeded  the  width  of  the  hardened  shelters  currently  available 
in  Europe,  the  rough  cost  to  build  new  shelters  would  be  determined. 
Though  only  a  rough  cost,  the  design  manager  could  use  the  estimate  to 
decide  whether  the  advantage  of  the  extra  width  was  a  sufficient  enough 
benefit  to  offset  the  additional  facility  cost  (Lollis,  1988). 

Force  Develooment/Force  Structure.  Similar  to  the  previous  two 
systems,  this  system  would  probably  be  limited  to  the  headquarters  level. 
The  system  would  could  be  used  in  three  different  modes:  for  planning,  for 
exercises,  and  for  actual  operations  (Cannon,  1988,  Showers,  1988). 

Developing  the  forces  to  go  to  war  is  now  done  largely  by  rule  of 
thumb.  Automating  it  with  an  expert  system  model  would  allow  many 
different  scenarios  to  be  played  out  before  the  actual  critical  deployment 
occurs.  Also,  the  system  could  include  information  such  as  exercise  and 
operational  experience  of  units  and  leaders  to  help  in  the  selection  of  units 
for  small  operations  (Cannon,  1988,  Showers,  1988). 
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V.  Summary,  Conclusions,  and  Recommendations 


Overview 

This  chapter  summarizes  the  research's  objective,  methodology  and 
findings,  emphasizing  conclusions  about  the  effort,  making 
recommendations  for  further  use  of  the  methodology,  and  suggesting 
further  research. 

Summary 

Research  Objective.  The  objective  of  this  thesis  was  to  search  for 
applications  for  expert  systems  within  civil  engineering.  To  accomplish  the 
objective,  a  methodology  was  developed  to  discern  which  knowledge  area 
had  potential  for  expert  system  development. 

Research  Methodology.  The  methodology  used  to  accomplish  the 

research  objective  was  to  follow  Waterman's  methodology,  with  some 
modification.  The  methodology  would  assure  that  the  problem  area 
selected  would  be  possible,  justified,  and  appropriate.  Eight  experienced 
civil  engineers  were  interviewed  during  two  rounds  of  questioning.  The 
first  round  solicited  and  screened  ideas,  using  the  developed  methodology, 
from  each  of  the  interviewees.  During  the  second  round  the  interviewees 
were  asked  to  select  the  ideas  they  felt  would  have  the  greatest  potential 
benefit  to  Air  Force  civil  engineering  in  rank  order. 

Research  Analysis.  The  first  round  of  interviews  generated  twenty- 

one  ideas.  Some  of  the  ideas  were  similar  and  were  combined  into  fifteen 
proposals  presented  in  the  second  round  of  interviews.  From  the  second 
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round  of  interviews,  five  proposals  emerged  having  the  greatest  potential 
for  benefitting  civil  engineering; 

1.  Job  Order/Work  Order  Management. 

2.  Design  Schedule  Management. 

3.  Beddown  of  New  Aircraft  Systems. 

4.  Facility  Constraints  on  New  Aircraft  Designs. 

5.  Force  Development/Force  Structure. 

Conclusions 

This  research  has  lead  to  several  conclusions. 

Conclusion  1:  Three  types  of  expert  systems  will  emerge  for  use  by 
Air  Force  Civil  Engineers:  Frequent  operations,  Infrequent  but  critical 
activities,  and  Training.  The  fifteen  proposals  from  the  first  round  of 
interviews  fall  into  these  application  categories,  shown  in  Table  4.  The 
application  categories  will  indicate  the  development  direction  of  the 
applications.  Frequent  operations  and  training  categories  will  be  used  Air 
Force  wide,  so  they  should  be  developed  on  WIMS  or  the  next  generation  of 
civil  engineering  computer  systems.  By  installing  the  expert  systems  on 
WIMS  or  the  next  generation,  the  users  will  have  easy  access  to  the  expert 
systems.  Infrequent  but  critical  operational  systems  should  be  developed  on 
a  personal  computers,  if  possible.  This  development  path  would  save  the 
the  capacity  on  the  WIMS  for  daily  usage. 

Conclusion  2:  Experts  and  users  need  to  be  part  of  the  final  selection 
process.  The  uncertainty  evident  in  several  of  the  responses  indicated  the 
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Table  4  -  Application  Categories 


Frequent  Operations- 

Job  Order/Work  Order  Management 
Design  Schedule  Management 
Vehicle  Allocation 
Energy  Management 

Infrequent  But  Critical  Operations: 

Force  Beddown  at  a  Bare  Base 
Selection  of  a  Minimum  Operating  Strip  (MOS) 
Force  Development/Force  Structure 
Beddown  of  New  Aircraft  Systems 
Facilities  Constraints  on  New  Aircraft  Designs 
Groundwater  Decontamination 
Well-Rounded  People  Advisor 
Scheduling  and  Assignment  of  Engineers 

Training 

Classification  of  Job  Orders/Work  Orders 

Corrosion  Control 

Liquid  Fuel  System  Maintenance 
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need  for  more  information  from  the  experts  and  the  users  before  the  final 
commitment  of  resources  to  developing  an  expert  system.  The  experts 
would  clarify  most  of  the  uncertainty  identified  by  the  engineering 
managers.  Users  could  define  the  requirements  of  the  problems  that  need 
to  be  solved  by  the  expert  system.  As  stated  in  Chapter  2,  user  involvement 
is  key  to  making  the  system  acceptable  and  workable.  Prerau  s  selection 
criteria  should  be  followed  during  the  final  selection  process  to  generate 
commitment  and  assure  that  no  political  problems  pose  an  impediment  to 
the  development  of  the  proposed  expert  system  (Prerau,  1985:  26-30). 

Conclusion  3:  Attempting  to  develop  the  expertise  may  be  as 
important  as  developing  the  system.  It  was  apparent  many  of  the  proposed 
areas  for  expert  system  development  were  also  areas  which  the 
interviewees  wanted  better  defined.  Attempting  to  develop  expert  systems 
in  such  areas  as  job  order/work  order  management  and  design  schedule 
management  may  be  quite  difficult,  but  would  help  further  define  those 
areas  In  short,  an  expert  system  development  may  be  one  way  to 
concentrate  the  knowledge  for  use. 

Conclusion  4:  Air  Force  Civil  Engineering  should  anticipate  the 
future  and  plan  for  expert  systems.  As  much  as  spreadsheets  and  databases 
are  a  part  of  our  computer  ability  now,  expert  systems  will  be  a  part  of  our 
future.  To  adequately  anticipate  using  expert  systems  in  the  future,  Air 
Force  civil  engineering  needs  to  plan  now  for  expert  systems.  Planning 
includes  providing  expert  system  tools  and  shells  for  our  expert  system 
development  on  our  next  generation  of  computers.  CE's  computers  should 
also  be  able  to  use  expert  systems  developed  by  other  sources  such  as  the 
Environmental  Protection  Agency,  U.S.  Army  Construction  Engineering 
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Research  Laboratory,  and  commercial  products.  A  pool  of  expertise, 
engineers  knowledgeable  in  expert  systems,  needs  to  be  nurtured  within 
Civil  Engineering,  so  civil  engineers  can  develop  their  own  systems. 

Conclusion  5:  The  search  for  possible  expert  systems  is  not 
completed.  Many  of  the  ideas  and  proposals  presented  in  this  thesis  would 
make  good  expert  systems;  but  many  other  problem  areas  have  not  been 
discussed.  Air  base  operability  is  growing  as  a  knowledge  area,  which 
might  in  the  near  future  have  a  great  potential  and  need  for  expert  systems 
(Cannon,  1988).  As  new  problems  arise,  so  will  the  potential  for  new  expert 
systems. 

Recommendations  on  the  Selection  Criteria 

The  methodology  worked  well,  though  several  changes  would 
improve  it.  Consideration  should  be  given  to  eliminating  unnecessary 
questions,  adding  scales  to  some  questions,  and  defining  a  manageable  size. 
Eliminate  Unnecessary  Questions.  In  retrospect  Questions  9  and  1 1 

were  actually  answered  by  other  questions,  and  are  probably  unnecessary. 
Question  9,  concerning  whether  experts  were  better  than  novices,  was 
answered  yes'  in  every  incidence.  The  more  discerning  question  is 
Question  7,  "Do  genuine  experts  exist?”,  which  was  answered  maybe'  three 
times.  Also  considered  unnecessary  was  Question  1 1,  concerning  the 
practical  value  of  the  expert  system  proposed.  The  practical  value  of  the 
system  is  actually  answered  by  the  justification  section. 

Scales.  The  unexpected  level  of  uncertainty  in  many  of  the  answers 

points  to  the  need  to  have  scales  for  four  of  the  questions.  The  scale  would 
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help  to  define  the  gray  area  in  between  yes  and  no  for  both  the  researcher 
and  the  interviewee. 

Question  2,  concerning  the  volatility  of  the  knowledge  area  was 
answered  yes  four  times  and  maybe'  once.  (  No  is  the  acceptable  answer 
in  this  case.)  Interestingly,  three  of  the  times  it  was  answered  yes',  the 
same  idea  was  answered  no'  by  another  interviewee,  showing  diversity  in 
opinion.  In  all  three  of  the  incidents,  Question  8  concerning  the  agreement 
between  experts  was  also  answered  maybe'. 

Question  12  also  gave  several  of  the  interviewees  difficulty  in 
answering  either  yes'  or  'no'.  Identifying  the  relative  percentage  of 
heuristic  solutions  and  percentage  algorithmic  solutions  would  give  the 
researcher  a  better  indication  whether  an  expert  system  is  really 
appropriate  or  not.  Question  16,  concerning  how  well  a  task  is  understood 
would  also  be  better  suited  by  a  scaled  response.  Scaling  these  questions  in 
the  future  would  measure  the  level  of  uncertainty  felt  by  the  interviewees. 

The  scales  could  be  presented  to  the  interviewees  in  terms  of 
percentage.  For  example,  using  Question  2: 

Is  the  task  in  a  volatile  knowledge  area?  What  percentage  of  the 
knowledge  area  is  stable  and  mature? 

a.  100%  b.  95%  c.  90%  ’d.  85%  e.  80%  or  less 

Scaling  Questions  8, 12,  and  16  in  a  similar  manner  would  also 
provide  the  researcher  with  a  better  feel  for  the  subject  in  consideration. 
Ultimately,  the  scaled  questions  in  concert  with  the  other  questions  would 
indicate  potential  for  expert  system  development.  Of  course,  proposals  with 
overwhelming  justification  could  be  attempted  in  the  interest  of  furthering 
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the  knowledge  area  even  if  the  expert  system  is  on  the  low  end  of  the 
potential  development  scale. 

Manageable  Size.  Describing  a  manageable  size  was  a  very  nebulous 
concept.  Attempting  to  ask  the  interviewees  directly  often  lead  to  some 
very  nebulous  answers.  Further  research  should  be  attempted  to  discern 
some  method  for  asking  this  question.  The  best  solution  would  be  a 
heuristic  identifying  a  manageable  size. 

Further  Research 

A  number  of  different  paths  might  extend  from  this  work.  Initially, 
the  selection  methodology  might  be  used  to  survey  civil  engineering  Air 
Force  wide.  The  selection  methodology  might  also  be  a  starting  point  for 
the  final  selection  and  development  of  some  of  the  proposals  suggested. 
Ultimately,  each  of  the  proposals  suggested  might  be  a  research  topic  or 
several  research  topics  in  themselves. 
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Appendix  A:  Definitions 


Artificial  Intelligence  -  A  field  aimed  at  pursuing  the  possibility  that  a 
computer  can  be  made  to  behave  in  ways  that  humans  recognize  as 
intelligent'  behavior  in  each  other  (Harmon  and  King,  1985:257). 

Case  -  A  particular,  specific  problem  for  which  an  expert  system  can 
perform  its  task.  Cases  are  used  by  the  knowledge  engineers  in 
developing,  expanding  and  evaluating  the  performance  of  expert 
systems  (McCain,  1987:  117). 

Decision  Support  System  (DSS)  -  An  interactive  system  that  helps  managers 
utilize  data  and  models  to  solve  unstructured  problems  (Holt,  1987). 

Domain  Expert  -  A  person  who  through  years  of  training  and  experience  has 
become  extremely  proficient  at  problem  solving  in  a  particular  domain 
(Waterman,  1986: 10). 

Expert  Systems  -  Any  computer  system  developed  by  means  of  an  expert- 
system-building  tool  mapping  an  expert  or  experts  knowledge  and 
expertise  (Harmon  and  King,  1985:  259). 

Expert-system-building  tool  -  The  programming  language  and  support 
package  used  to  build  an  expert  system  (Waterman,  1986: 1 1). 

Heuristic  -  A  rule  of  thumb  or  other  device  or  simplification  that  reduces  or 
limits  search  in  a  problem  area.  Heuristics  differ  from  algorithms  in 
that  heuristics  do  not  always  insure  a  correct  answer  (Harmon  and 
King,  1985:  260). 

Inference  -  The  process  by  which  new  facts  are  derived  from  known  facts. 

A  rule  (e.g.,  If  the  sky  is  black,  then  the  time  is  night),  combined  with  a 
rule  of  inference  (e.g.,  Modus  ponens)  and  a  known  fact  (e.g.,  The  night 
is  black)  results  in  a  new  fact  (e.g.,  The  time  is  night)  (Harmon  and 
King,  1985:  261). 

Knowledge  Engineer  -  An  individual  specializing  in  analyzing  problems, 
acquiring  knowledge,  and  constructing  expert  systems  (Harmon  and 
King,  1985:  262). 
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Management  Information  System  (MIS)  -  a  system  that  accumulates,  stores, 
processes,  and  transmits  information  (Holt,  1987). 

Modus  ponens  -  A  rule  of  logic,  asserting  that  if  A  implies  B,  and  A  is  true, 
then  B  is  true  (Harmon  and  King,  1985:  263). 

User  -  A  person  who  uses  an  expert  system,  such  as  domain  expert,  a 
knowledge  engineer,  clerical  staff,  or  anyone  that  uses  the  developed 
system  (Waterman,  1986;  10-11). 


Appendix  B;  Prerau  s  Selection  Criteria 
for  an  Appropriate  Domain  for  an  Expert  System 


BASIC  REQUIREMENTS 

-  The  domain  is  characterized  bv  the  use  of  expert  knowledge- 
judgment.  and  experience.  The  goal  of  the  project  is  to  extract  a  portion  of 
an  expert  s  knowledge,  judgment  and  experience,  and  put  it  in  a  program. 

-  Conventional  programming  (algorithmic)  approaches  to  the  task  are 
not  satisfactory.  If  a  conventional  approach  will  work  well,  there  is  usually 
less  technical  risk  to  using  it  rather  than  an  expert  system  approach.  Note, 
however,  that  expert  system  methodology  may  offer  some  additional 
advantages  over  conventional  techniques,  such  as  the  expected  ease  of 
updating  and  maintaining  a  knowledge  base  and  the  ability  to  explain 
results. 

-  There  are  recognized  experts  that  solve  the  problem  today.  If  an 
area  is  too  new  or  too  quickly  changing,  there  may  be  no  real  experts. 
However,  these  are  often  the  areas  that  are  suggested  for  expert  system 
developments. 

-  The  experts  are  probably  better  than  amateurs  in  performing  the 
task.  Thus,  the  task  does  require  expertise. 

-  Expertise  is  not  or  will  not  te  available,  on  a  reliable,  and  continuing 
basis,  i.e..  there  is  a  need  to  'capture"  the  expertise.  Thus,  there  is  a  need 
for  the  expert  system.  For  example:  ( 1 )  expertise  is  scarce,  (2)  expertise  is 
expensive,  (3)  there  is  a  strong  dependence  on  overworked  experts,  and/or 
(4)  expertise  is  available  today,  but  will  be  unavailable,  or  less  available,  in 
the  future. 

-  The  completed  system  is  expected  to  have  a  significant  pavoff  for 
the  corporation. 

-  Among  possible  application  domains,  the  domain  selected  is  that  one 
that  best  meets  overall  project  goals  regarding  project  payoff  versus  risk  of 
failure.  For  example,  a  conservative  approach  would  be  to  attempt  to 
develop  a  system  that  would  meet  some  criterion  for  minimum  payoff  if 
successful,  and  that  seems  to  offer  the  best  chance  of  success. 
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TYPE  OF  PROBLEM 


-  The  task  primarily  requires  symbolic  reasoning.  For  a  task  that 
primarily  involves  numerical  computation,  consideration  should  also  be 
given  to  other  programming  approaches. 

-  The  task  requires  the  use  of  heuristics,  e.g..  rules  of  thumb, 
strategies,  etc.  It  may  require  consideration  of  an  extremely  large  number 

of  possibilities  or  it  mav  require  decisions  to  be  based  upon  incomplete  or 

uncertain  information.  A  strength  of  expert  systems  is  their  ability  to 
handle  heuristics.  Problems  with  very  large  numbers  of  possibilities  or 
with  incomplete  or  uncertain  information  are  difficult  to  attack  by 
conventional  approaches,  but  may  be  amenable  to  expert  system 
methodologies. 

-  The  system  development  has  as  its  goal  either  to  develop  a  system 
for  actual  use  or  to  make  major  advances  in  the  state  of  the  art  of  expert 
system  technology,  but  does  not  attempt  to  achieve  both  of  these  goals 
simultaneously.  Doing  both  simultaneously  is  laudable,  but  more  difficult. 

-  The  task  is  defined  very  clearly:  At  the  project  outset,  there  should 
be  a  precise  definition  of  the  incuts  and  outputs  of  the  system  to  be 
developed.  This  is  a  good  attribute  of  any  task.  However,  it  is  not 
necessary  that  the  task  definition  be  fixed  for  all  time.  As  the  system 
evolves  and  task  situations  change,  it  should  be  possible  to  change  the  task 
definition  accordingly. 

THE  EXPERT 

-  There  exists  an  expert  to  work  with  the  project.  This  is  the  source 
of  expertise. 

-  The  expert  s  knowledge  and  reputation  must  be  such  that  if  the 
expert  system  is  able  to  capture  a  portion  of  the  expert's  expertise,  the 
system’s  output  will  have  credibility  and  authority.  Otherwise,  the  system 
may  not  be  used.  (This  may  not  be  necessary  in  a  domain  where  an 
accepted  test  for  "goodness"  of  result  exists.) 

-  The  expert  has  built  up  expertise  over  a  long  period  of  task 
performance.  Thus,  the  expert  has  had  the  amount  of  experience  necessary 
to  be  able  to  develop  the  insights  into  the  area  that  result  in  heuristics. 
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development  of  the  system.  This  is  often  a  problem.  The  best  experts,  in 
the  most  important  corporate  areas,  are  usually  the  ones  that  can  be  least 
spared  from  their  usual  position. 

-  The  expert  is  capable  of  communicating  his  knowledge,  judgment. 

experience,  and  the  methods  used  to  apply  them  to  the  particular  task. 

It  is  important  to  find  an  expert  that  has  not  only  the  expertise,  but  also  the 
ability  to  impart  it  to  the  project  team,  whose  members  probably  know  little 
or  nothing  about  the  subject  area.  The  expert  should  be  able  to  introspect 
to  analyze  reasoning  process  clearly  to  the  project  team,  and  to  discuss  it 
with  them. 

-  The  expert  is  cooperative.  The  expert  should  be  eager  to  work  on 
the  project  or,  at  worst,  nonantagonistic. 

-  The  expert  should  be  easy  to  work  with.  The  project  team  and  the 
expert  will  be  spending  a  lot  of  time  together. 

-  The  expertise  for  the  system,  at  least  that  pertaining  to  one 
particular  sub-domain,  is  to  be  obtained  primarily  from  one  expert.  This 
avoids  the  problem  of  dealing  with  multiple  experts  whose  conclusions  or 
problem-solving  techniques  do  not  agree.  However,  there  may  be  some 
advantages  to  using  multiple  experts--e.g.,  strength  of  authority  and 
breadth  of  expertise  in  sub-domains. 

-  ILmultiple  experts  contribute  in  a  particular  sub-domain,  one  of 
them  should  be  the  primary  expert  with  final  authority.  This  allows  all  the 
expertise  to  be  filtered  through  a  single  person's  reasoning  process.  (Note 
that  some  techniques  have  been  developed,  in  disciplines  such  as  economic 
modeling  and  technological  forecasting,  to  allow  combining  inputs  from 
multiple  experts.) 

PROBLEM  BOUNDS 

-  The  task  is  neither  too  easy  (taking  a  human  expert  less  than  a  few 
minutes)  nor  too  difficult  (requiring  more  that  a  few  hours  for  an  expert).  If 
the  task  is  too  easy,  the  development  of  the  system  may  not  warrant  the 
effort:  if  too  difficult,  the  amount  of  knowledge  needed  may  be  beyond  the 
state  of  the  art  in  knowledge  base  size. 
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make  the  knowledge  base  developed  interesting.  If  it  is  too  small,  the  task 
may  be  more  amenable  to  another  approach— e.g.,  a  decision  tree. 

-  The  task  is  sufficiently  narrow  and  self-contained.-  the  aim  is  not 
for  a  system  that  is  expert  in  an  entire  domain,  but  for  a  system  that  is  an 
eioert  in  a  limited  task  within  the  domain.  This  more  tightly  bounds  the 
task,  which  should  help  keep  the  size  of  the  knowledge  base  bounded. 

-  The  number  of  important  concents  (e.g..  rules)  required  is  bounded 
to  several  hundreds.  This  is  a  reasonable  size  for  an  expert  system,  though 
the  number  can  go  into  the  thousands. 

DOMAIN  AREA  PERSONNEL 

-  Personnel  in  the  domain  area  are  realistic,  understanding  the 
potential  of  an  expert  system  for  their  domain,  but  also  realizing  that  thus 

far  few  expert  systems  have  resulted  in  actual  production  programs  with 

major  industrial  oavoff.  The  system  recipients  should  not  be  overly 
pessimistic.  The  project  team  may  have  to  educate  them  to  understand 
what  are  reasonable  expectations. 

-  Domain  area  personnel  understand  that  even  a  successful  system 
will  likely  be  limited  in  scope  and,  like  a  human  expert,  may  not  produce 
optimal  or  correct  results  100%  of  the  time.  The  expert  system  will 
probably  be  no  better  than  a  limited  version  of  the  expert— this  must  be 
enough. 

-  There  is  strong  managerial  support  from  the  domain  area,  especially 
regarding  the  large  commitment  of  time  bv  the  experts(s).  and  their 
possible  travel  or  temporary  relocation,  if  required.  This  should  all  be 
agreed  upon  up  front. 

-  The  specific  task  within  the  domain  is  jointly  agreed  upon  bv 
system  developers  and  the  domain  area  personnel.  This  helps  ensure  that 
the  system,  if  successful,  will  be  useful  and  will  be  used. 

-  Managers  in  the  domain  area  have  previously  identified  the  need  tc 
solve  the  problem  which  the  system  attacks.  This  is  strong  evidence  that 
the  system  is  needed  and  makes  managerial  support  more  likely. 
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-  The  project  is  strongly  supported  bv  a  senior  manager,  for 
protection  and  follow-uo. 

-  Potential  users  would  welcome  the  completed  system.  If  not,  will 
the  system  ever  be  used?  The  project  team  should  consider  how  to  make 
the  system  unthreatening  to  the  users  and  welcomed  by  them. 

-  The  system  can  be  introduced  with  minimal  disturbance  of  the 
current  practice.  This  will  make  the  users  acceptance  of  the  system  more 
likely. 


-  The  user  group  is  cooperative  and  patient. 

-  The  introduction  of  the  system  will  not  be  politically  sensitive  or 
controversial.  If  not,  the  potential  resulting  problems  should  be  considered 
in  advance.  One  typical  problem:  The  control  or  use  of  the  system  goes 
across  existing  organizational  boundaries. 

-  The  knowledge  contained  bv  the  system  will  not  be  politically 
sensitive  or  controversial.  For  example,  there  may  be  certain  practices, 
embodied  in  heuristics,  which  may  prove  embarrassing  if  written  down, 
such  as  how  certain  customers  are  treated  relative  to  other  customers. 

-  The  system  s  results  will  not  be  politically  sensitive  or  controversial. 
If  there  will  be  corporate  parties  who  will  challenge  the  system  if  its  results 
do  not  favor  them  politically  (e.g.,  on  appropriation  of  funds),  then  it  will  be 
much  harder  to  gain  system  acceptance. 

OTHER  DESIRABLE  FEATURES 

-  Ihe_ivstem  can  be  phased  into  use  gracefully:  Some  percentage  of 
incomplete  coverage  can  be  tolerated  (at  least  initially),  and  the 
determination  of  whether  a  sub-problem  is  covered  bv  the  present  system 
is  not  difficult  If  the  system  does  not  have  to  do  everything  in  order  to  do 
something,  it  can  be  put  in  place  much  sooner.  The  more  difficult  problems 
can  be  solved  later,  if  at  all. 

-  The  task  is  decomposable,  allowing  relatively  rapid  prototyping  for 
a  closed  small  subset  of  the  complete  task;  and  then  slow  expansion  to  the 
complete  task.  This  makes  development  much  easier. 
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-  The  task  is  not  all-or-nothing:  Some  percentage  of  incorrect  or 
nonootimal  results  can  be  tolerated.  The  more  toleration  for  incorrect 
results,  the  faster  the  system  can  be  deployed  and  the  easier  it  wilt  be  to 
win  system  acceptance.  For  example,  in  a  domain  where  even  the  best 
experts  are  often  wrong,  system  users  will  not  be  as  upset  by  an  incorrect 
result  from  the  system. 

-  The  skill  required  bv  the  task  is  taught  to  novices.  Thus,  the  task  is 
not  "unteachable,"  and  there  is  some  experience  with  teaching  the  domain 
knowledge  to  neophytes,  such  as  the  project  team  (and  ultimately,  the 
system).  Furthermore,  this  usually  means  that  there  is  an  organization  to 
the  knowledge  that  can  prove  useful  (at  least  initially)  in  building  the 
system. 

-  There  are  books  or  other  written  materials  discussing  the  domain. 

If  this  is  true,  then  an  expert  has  already  extracted  and  organized  some  of 
the  domain  expertise.  As  in  the  previous  point,  this  organized  knowledge 
might  prove  useful  (at  least  initially)  in  building  the  system.  Note,  however, 
that  one  benefit  of  capturing  an  expert  s  domain  knowledge  might  be  to 
make  a  step  toward  formalizing  a  domain  that  has  not  been  treated  in  a 
formal  manner  before. 

-  The  task's  payoff  is  measurable.  If  not,  it  is  harder  to  demonstrate 
success  to  skeptics. 

-  Experts  would  agree  on  whether  the  system's  results  are  good 
(correct).  If  not  the  system's  results  are  open  to  challenge,  even  if  the 
system  accurately  embodies  the  expert's  knowledge. 

-  Test  cases  are  available.  This  make  development  much  easier. 

-  The  need  for  the  task  is  projected  to  continue  for  several  years. 

The  need  must  exist  enough  beyond  the  period  of  system  development  to 
generate  the  payoff. 

-  The  domain  is  fairly  stable.  Expected  changes  are  such  that  they 
utilize  the  strengths  of  expert  systems  (e.g..  ease  of  updating  or  revising 
specific  rules  in  a  knowledge  base,  but  will  not  require  major  changes  in 
reasoning  processes.  An  unstable  domain  may  yield  a  situation  where  a 
large  number  of  previously  developed  knowledge  structures  (e.g.,  rules)  are 
no  longer  valid  but  cannot  easily  be  change  without  redoing  the  entire 
development  process. 
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expected  to  be  pursued.  However,  if  a  project  goal  is  to  compare  expert 
system  technology  to  other  technologies  this  may  be  just  what  is  desired. 


I  iH'l  UK-IM  dim I  iTTOil  1  §  ilMrTlVfQ  1  PEH  W'JHMit  I 


and  has  no  absolute  milestones  for  completion.  The  use  of  expert  system 
technology  for  real  corporate  applications  is  still  relatively  new,  and  so  any 
development  has  some  risk.  Thus,  the  less  dependent  other  activities  are, 
the  better. 


This  gives  good  promise  of  project  success. 
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This  also  makes  success  more  likely. 


effort.  Though  it  is  certainly  possible  to  develop  a  system  for  a  problem 
with  a  real-time  requirement,  the  considerations  involved  divert  effort  from 
the  primary  task  knowledge  acquisition. 

-  The  user  interface  will  not  require  extensive  effort.  As  with  a  real¬ 
time  requirement,  if  the  work  required  is  excessive,  it  could  divert  effort 
from  knowledge  acquisition. 

Source:  David  S.  Prerau,  "Selection  of  an  Appropriate  Domain  for  an  Expert 
System,"  The  AI  Magazine.  Summer,  1985,  pg.  27-30. 
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Appendix  G  Preliminary  Selection  Criteria 


This  preliminary  selection  criteria  provides  guidance  and  information 
concerning  the  development  of  expert  systems.  It  is  to  be  used  by  senior 
management  in  reviewing  problem  areas  that  might  be  helped  by  an  expert 
system.  After  successfully  screening  a  possible  application,  the  expert  or 
experts  and  the  users  should  become  involved  with  the  final  selection,  using 
Prerau's  selection  criteria  (Prerau,  1985:  26-30). 

Typically,  all  the  answers  to  this  preliminary  selection  criteria  must  be 
yes  (except  of  the  Justified  section  and  questions  2  &  15),  to  recommend 
proceeding  with  the  development  of  an  expert  system.  The  justified  section 
requires  only  one  yes  response. 

Realistic _ _ 

1 .  Will  the  expert  system  tackle  problems  managers  ha  ve  tackled  before? 
If  the  problem  currently  cannot  be  solved,  an  expert  system  will  not 
help  (Cross,  1988). 

2.  Is  the  task  in  a  volatile  knowledge  area?  Because  of  the  development 
time,  expert  systems  are  generally  developed  only  in  mature  knowledge 
areas  (Cross,  1988). 

Justified _ 

3.  Does  the  task  have  a  high  payo/T?  The  extensive  development  costs 
must  have  a  substantial  offset  or  payoff  to  be  a  justified  venture  for  an 
organization  (Waterman,  1986:  130). 

4.  Is  the  human  expertise  scarce  or  being  lost?  The  expert  system  can  be 
justified  scarce  expertise  or  expertise  being  lost  (Waterman,  1986.  130). 

5.  Is  the  expertise  needed  in  many  locations  at  once?  Expert  systems  can 
easily  be  reproduced  to  be  used  at  many  locations  at  once  (Waterman, 
1986:  130). 

6.  Is  the  expertise  needed  in  a  hostile  environment?  Expert  systems  can 
be  risked  in  environments  of  high  risk,  without  risking  the  experts 
(Waterman,  1986:  130). 
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Expertise 


Expert  Characteristics 

.  .  experts  are  not  only  proficient,  but  smooth  and  efficient  in  the 

actions  they  take"  (Johnson,  1983;  78). 

"Real  experts  not  only  produce  good  solutions,  but  often  find  them 

quickly,  while  novices  tend  to  take  much  longer  to  find  the  same  solutions" 

(Waterman,  1986;  25). 

7.  Do  genuine  experts  exist?  If  genuine  experts  do  not  exist,  development 
of  the  system  may  be  very  difficult  or  impossible  (Waterman,  1986: 

129). 

8.  Do  the  experts  agree  on  solutions?  If  the  experts  do  not  agree  on  the 
solutions,  then  the  expert  system  may  be  of  little  or  no  use  (Waterman, 
1986:  130). 

9.  Are  experts  better  than  novices  at  performing  the  task?  If  a  novice  can 
perform  as  well  as  an  expert,  then  there  is  probably  no  advantage  in 
developing  an  expert  system  (Prerau,  1985:  27). 

1 0.  Can  the  eiperts  can  articuiate  their  methods?  Ultimately,  the  experts’ 
methods  must  be  represented  by  the  knowledge  base  within  the  expert 
system.  If  the  experts  cannot  articulate  their  methods,  the  knowledge 
base  cannot  be  constructed  (Waterman,  1986: 129). 

Task _ 

1 1.  Does  the  task  have  a  practical  value?  The  task  should  cover  enough 
information  that  users  would  be  interested  in  using  an  expert  system  for 
assistance  (Waterman,  1986:  132). 

1 2.  Does  the  task  use  heuristic  solutions  versus  algorithms?  Heuristic 
solutions  are  based  on  strategies  and  rules  of  thumb,  based  on 
incomplete  or  uncertain  information.  Algorithmic  solutions  use 
exhaustive  methods  based  on  theoretical,  mathematical,  and/or 
empirical  techniques.  Expert  systems  work  best  with  heuristic 
information  (Waterman,  1986;  130). 

13.  Can  the  task  be  manipulated  symbolically?  This  deals  with  the  type  of 
problem.  Another  way  to  consider  the  same  question  is  the  telephone 
test.  Could  an  expert,  given  the  necessary  time,  guide  a  novice  through 
the  problem?  If  so,  the  task  could  probably  manipulated  symbolically 
(Waterman,  1986:  130). 
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1 4.  Do  the  problems  encountered  share  some  of  the  same  common 
characteristics?  Problem  that  the  expert  system  will  solve  should  have 
some  common  characteristics  or  data  inputs  to  the  system.  If  they  do 
not,  the  development  of  an  expert  system  will  be  very  difficult  (Cross, 
1988). 

15.  Are  conventional  programming  techniques  satisfactory?  Expert 
systems  are  often  thought  of  as  a  last  resort  because  of  the  extensive 
time  and  cost  invested  into  the  development  effort.  If  no  other 
programming  technique  will  do  all  that  is  required  or  come  close,  then 
an  expert  system  may  be  the  answer  (Prerau,  1985-  27). 

16.  Is  the  task  is  weii understood?  This  question  complements  Question 
10.  "If  the  task  is  so  new  or  so  poorly  understood  that  it  requires  basic 
research  to  find  solutions,  knowledge  engineering  will  not  work 
(Waterman,  1986:  129). 

17.  Can  the  skill  required  can  be  taught  to  novices?  If  the  skill  cannot  be 
taught  to  novices,  it  is  unlikely  that  it  can  be  developed  into  an  expert 
system  (Prerau,  1985.  29). 

18.  Is  the  task  is  of  a  manageable  size?  The  task  should  be  sufficiently 
narrow  and  self-contained.  Another  way  to  think  about  it  is,  the  expert 
system  should  be  the  eipert  for  a  limited  task  within  the  knowledge 
domain  (Waterman,  1986;  132). 

1 9.  Are  cases  are  available  to  develop  and  verify  the  validity  of  the  system? 
The  development  and  the  verification  of  the  expert  system  depend  on 
the  having  cases  illustrating  the  problem  encountered  to  the  knowledge 
engineer  (Prerau,  1985:  29). 

Other  Considerations _ 

20.  Is  the  task  difficult  enough  to  require  an  expert?  The  problem  should 
not  be  too  easy,  otherwise  the  knowledge  could  be  transferred  directly 
to  the  user  (Waterman,  1986:  132). 

21.  Does  the  task  require  commonsense?  Expert  systems  do  not  aeal  well 
with  commonsense  reasoning  problems  (Waterman,  1986:  129). 

22.  If  commonsense  is  required,  can  the  user  provide  it  ?  If  the  user  can 
provide  the  commonsense  required  for  the  problem,  then  the  expert 
system  would  be  of  use  (Waterman,  1986:  129). 

23.  Does  the  task  require  cognitive  skills  on/y?\l  senses,  such  a  refined 
sense  of  smell,  are  required,  then  the  user  will  have  to  supply  these 
skills  (Waterman,  1986:  129). 
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Appendix  D;  List  of  Interviewees 


Col  George  E.  Cannon  Jr.,  Dean,  School  of  Civil  Engineering,  Air  Force 
Institute  of  Technology  (AU),  Wright-Patterson  AFB  OH 

Col  Joe  L.  Hicks,  Director  of  Operations  and  Maintenance,  Engineering  and 
Services,  Headquarters  Air  Force  Logistics  Command,  Wright- 
Patterson  AFB  OH 

Col  Thomas  E.  Lollis,  Deputy  Chief  of  Staff  for  Acquisition  Civil 

Engineering,  Aeronautical  Systems  Division,  Wright-Patterson  AFB 
OH 

Col  Nicholas  A.  Scambilis,  Base  Civil  Engineer,  Wright-Patterson  AFB  OH 

Col  James  G.  Zody,  Assistant  Deputy  Chief  of  Staff,  Engineering  and 
Services,  Headquarters  Air  Force  Logistics  Command.,  Wright- 
Patterson  AFB  OH 

Maj  Mark  N.  Goltz,  Department  Head,  Department  of  Management 
Applications,  School  of  Civil  Engineering,  Air  Force  Institute  of 
Technology  (AD),  Wright-Patterson  AFB  OH 

Maj  Duncan  H.  Showers,  Instructor, Department  of  Management 

Applications,  School  of  Civil  Engineering,  Air  Force  Institute  of 
Technology  (AU),  Wright-Patterson  AFB  OH 

Maj  Timothy  G.  Wise,  Executive  Officer  to  the  Deputy  Chief  of  Staff, 
Engineering  and  Services,  Headquarters  Air  Force  Logistics 
Command,  Wright-Patterson  AFB  OH 
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the  future.  Air  Force  Civil  Engineering  will  be  increasingly 
concerned  with  its  productivity  and  effectiveness.  This 
thesis  searched  for  Air  Force  Civil  Engineering  expert  system 
applications  using  a  preliminary  selection  criteria  to  discern 
the  knowledge  areas  having  expert  system  potential.  Interviews 
were  conducted  with  experienced  civil  engineers  to  gather  the 
ideas . 

The  primary  objective  of  this  thesis  was  to  develop  a 
preliminary  selection  criteria.  Donald  Waterman's  selection 
criteria  was  used  as  the  basis.  The  questions  within  the 
selection  criteria  were  reordered  with  the  most  discriminating 
questions  first,  to  eliminate  unfruitful  ideas  quickly.  Other 
discriminating  questions  were  added  to  the  selection  criteria 
as  necessary  for  clarification  and  amplification. 

Eight  experienced  civil  engineers  were  interviewed  during 
two  rounds  of  questioning.  The  first  round  of  interviews 
solicited  and  screened  ideas,  using  the  preliminary  selection 
criteria.  The  first  round  generated  twenty-one  ideas,  which 
were  combined  into  fifteen  proposals.  In  the  second  round, 
interviewees  selected  proposals  having  the  greatest  potential 
benefit  to  Air  Force  Civil  Engineering  in  order.  Five 
proposals  emerged  from  the  second  round  of  interviewing: 

Job  Order /Work  Order  Management,  Design  Schedule  Management, 
Beddown  of  New  Aircraft  Systems,  Facility  Constraints  on  New 
Aircraft  Designs,  and  Force  Development/Force  Structure. 
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